How to make an android remote control for PC

There are a lot of apps out there with which you can control your PC in various different ways. This article tries to explain how is it possible to write such apps. We’ll try to build a very simple remote for VLC media player. Using this remote, the user will be able to play/pause a video or jump to next/previous video in the playlist and also move the mouse around the screen.

To control our PC using an android app, we need a piece of code to run the server side (PC) to executecommands sent from the client side (android app) code. First, we’ll write the client side android app and then the server side java code.

CREATING CLIENT APP

As this example is meant for only for explaining the concept, we’ll create a very simple one page android app. It has only one activity which has 4 controls. One TextView which will act as our mouse pad and 3 buttons (Next, Play/Pause, Previous).

Create a new android project by the name RemoteDroid, with a blank activity (activity_main). Create the required textview and buttons either using the design view or text view. You can copy paste the following code to create the required activity layout.

We also need a button which will help us initiate connection to the server. We’ll represent this button in the action bar with “ic_action_cast” icon that comes with android Action Bar Icon Pack. Add this icon in your drawable folder and this line in your main menu (menu_main.xml under res->menu).

<item android:id=”@+id/action_connect” android:title=”@string/action_connect”

android:icon=”@drawable/ic_action_cast” app:showAsAction=”ifRoom” />

Finally the app should look like this:-

To send data to server, we need to create a stream socket and connect it to pre-determined port number and server IP address.

InetAddress serverAddr = InetAddress.getByName(“SERVER_IP”);

Socket socket = new Socket(serverAddr, PORT);

Once, the socket is connected, we need to create an output stream on that socket to send data to the server. Anything that we write on that output stream will be sent to the server.

PrintWriter outStream = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket.getOutputStream())), true);

outStream.println(“Hi”) //This will send “Hi” to server

When the user taps on play, next or previous button, we send “play”, “next” or “previous” text respectively to the server using outStream.println() and when the user drags on the mousePad text view, we send the movement in x and y direction in “x,y” format. To implement this complete functionality, paste the following code in your MainActivity.java file

package in.codesmith.remotedroid; import android.content.Context; import android.os.AsyncTask; import android.support.v7.app.ActionBarActivity; import android.os.Bundle; import android.util.Log; import android.view.Menu; import android.view.MenuItem; import android.view.MotionEvent; import android.view.View; import android.widget.Button; import android.widget.TextView; import android.widget.Toast; import java.io.BufferedWriter; import java.io.IOException; import java.io.OutputStreamWriter; import java.io.PrintWriter; import java.net.InetAddress; import java.net.Socket; public class MainActivity extends ActionBarActivity implements View.OnClickListener { Context context; Button playPauseButton; Button nextButton; Button previousButton; TextView mousePad; private boolean isConnected=false; private boolean mouseMoved=false; private Socket socket; private PrintWriter out; private float initX =0; private float initY =0; private float disX =0; private float disY =0; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); context = this; //save the context to show Toast messages //Get references of all buttons playPauseButton = (Button)findViewById(R.id.playPauseButton); nextButton = (Button)findViewById(R.id.nextButton); previousButton = (Button)findViewById(R.id.previousButton); //this activity extends View.OnClickListener, set this as onClickListener //for all buttons playPauseButton.setOnClickListener(this); nextButton.setOnClickListener(this); previousButton.setOnClickListener(this); //Get reference to the TextView acting as mousepad mousePad = (TextView)findViewById(R.id.mousePad); //capture finger taps and movement on the textview mousePad.setOnTouchListener(new View.OnTouchListener() { @Override public boolean onTouch(View v, MotionEvent event) { if(isConnected && out!=null){ switch(event.getAction()){ case MotionEvent.ACTION_DOWN: //save X and Y positions when user touches the TextView initX =event.getX(); initY =event.getY(); mouseMoved=false; break; case MotionEvent.ACTION_MOVE: disX = event.getX()- initX; //Mouse movement in x direction disY = event.getY()- initY; //Mouse movement in y direction /*set init to new position so that continuous mouse movement is captured*/ initX = event.getX(); initY = event.getY(); if(disX !=0|| disY !=0){ out.println(disX +","+ disY); //send mouse movement to server } mouseMoved=true; break; case MotionEvent.ACTION_UP: //consider a tap only if usr did not move mouse after ACTION_DOWN if(!mouseMoved){ out.println(Constants.MOUSE_LEFT_CLICK); } } } return true; } }); } @Override public boolean onCreateOptionsMenu(Menu menu) { // Inflate the menu; this adds items to the action bar if it is present. getMenuInflater().inflate(R.menu.menu_main, menu); return true; } @Override public boolean onOptionsItemSelected(MenuItem item) { // Handle action bar item clicks here. The action bar will // automatically handle clicks on the Home/Up button, so long // as you specify a parent activity in AndroidManifest.xml. int id = item.getItemId(); //noinspection SimplifiableIfStatement if(id == R.id.action_connect) { ConnectPhoneTask connectPhoneTask = new ConnectPhoneTask(); connectPhoneTask.execute(Constants.SERVER_IP); //try to connect to server in another thread return true; } return super.onOptionsItemSelected(item); } //OnClick method is called when any of the buttons are pressed @Override public void onClick(View v) { switch (v.getId()) { case R.id.playPauseButton: if (isConnected && out!=null) { out.println(Constants.PLAY);//send "play" to server } break; case R.id.nextButton: if (isConnected && out!=null) { out.println(Constants.NEXT); //send "next" to server } break; case R.id.previousButton: if (isConnected && out!=null) { out.println(Constants.PREVIOUS); //send "previous" to server } break; } } @Override public void onDestroy() { super.onDestroy(); if(isConnected && out!=null) { try { out.println("exit"); //tell server to exit socket.close(); //close socket } catch (IOException e) { Log.e("remotedroid", "Error in closing socket", e); } } } public class ConnectPhoneTask extends AsyncTask<String,Void,Boolean> { @Override protected Boolean doInBackground(String... params) { boolean result = true; try { InetAddress serverAddr = InetAddress.getByName(params[0]); socket = new Socket(serverAddr, Constants.SERVER_PORT);//Open socket on server IP and port } catch (IOException e) { Log.e("remotedroid", "Error while connecting", e); result = false; } return result; } @Override protected void onPostExecute(Boolean result) { isConnected = result; Toast.makeText(context,isConnected?"Connected to server!":"Error while connecting",Toast.LENGTH_LONG).show(); try { if(isConnected) { out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(socket .getOutputStream())), true); //create output stream to send data to server } }catch (IOException e){ Log.e("remotedroid", "Error while creating OutWriter", e); Toast.makeText(context,"Error while connecting",Toast.LENGTH_LONG).show(); } } } }

We can ask for server IP from the user in an EditText and then connect to that IP on a pre-determined port number, but for this example, we have hard-coded both these values and you should replace them based on your requirement.

WRITING THE SERVER

The client just connects via an open socket and send some text over to the server. Server is where all the magic of remote control happens. To simulate keyboard and mouse events on the server, we use java.awt.Robot class. This class was built mainly for the purpose of test automation, but can be used for various other purposes.

First thing we need to do on server is to open a server socket and then wait for a client to connect to it. Let’s create a server socket on a port

ServerSocket serverSocket = new ServerSocket(SERVER_PORT);

Socket clientSocket = server.accept();

Once the client is connected and is sending data, we’ll read the input line by line in a BufferedReader and then act on the basis of each input.

BufferedReader in = new BufferedReader(new InputStreamReader(client.getInputStream()));

String line = in.readLine();

To simulate keyboard events, we’ll the keyPress and keyRelease methods of Robot class. So for example, when the user wants to move to next song in the play list, he/she taps on “Next” button in the app which sends the string “next” to the server and upon receiving this string, the server simulates the press and release of key “n” on keyboard. We are using following VLC shortcuts:-

Next – n

Previous – p

Play/Pause – space

These are default shortcuts, you need to modify code if you have changed your VLC shortcuts.

Both keyPress and keyRelease methods take an integer as input. These integers can be obtained from KeyEvent class. KeyEvent has static fields corresponding to each keyboard key and event. This is how we simulate press and release of “n”

Robot robot = new Robot();

robot.keyPress(KeyEvent.VK_N);

robot.keyRelease(KeyEvent.VK_N);

To simulate a mouse event, we need to use the mouse related methods in Robot class. To move the mouse pointer, use Robot.mouseMove(x,y) and to simulate mouse click, use Robot.mousePress() and Robot.mouseRelease().

The complete server code can be written as below:-

import java.awt.AWTException; import java.awt.Robot; import java.awt.event.InputEvent; import java.awt.event.KeyEvent; import java.awt.MouseInfo; import java.awt.Point; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; import java.io.PrintWriter; import java.net.InetSocketAddress; import java.net.ServerSocket; import java.net.Socket; public class RemoteDroidServer { private static ServerSocket server = null; private static Socket client = null; private static BufferedReader in = null; private static String line; private static boolean isConnected=true; private static Robot robot; private static final int SERVER_PORT = 8998; public static void main(String[] args) { boolean leftpressed=false; boolean rightpressed=false; try{ robot = new Robot(); server = new ServerSocket(SERVER_PORT); //Create a server socket on port 8998 client = server.accept(); //Listens for a connection to be made to this socket and accepts it in = new BufferedReader(new InputStreamReader(client.getInputStream())); //the input stream where data will come from client }catch (IOException e) { System.out.println("Error in opening Socket"); System.exit(-1); }catch (AWTException e) { System.out.println("Error in creating robot instance"); System.exit(-1); } //read input from client while it is connected while(isConnected){ try{ line = in.readLine(); //read input from client System.out.println(line); //print whatever we get from client //if user clicks on next if(line.equalsIgnoreCase("next")){ //Simulate press and release of key 'n' robot.keyPress(KeyEvent.VK_N); robot.keyRelease(KeyEvent.VK_N); } //if user clicks on previous else if(line.equalsIgnoreCase("previous")){ //Simulate press and release of key 'p' robot.keyPress(KeyEvent.VK_P); robot.keyRelease(KeyEvent.VK_P); } //if user clicks on play/pause else if(line.equalsIgnoreCase("play")){ //Simulate press and release of spacebar robot.keyPress(KeyEvent.VK_SPACE); robot.keyRelease(KeyEvent.VK_SPACE); } //input will come in x,y format if user moves mouse on mousepad else if(line.contains(",")){ float movex=Float.parseFloat(line.split(",")[0]);//extract movement in x direction float movey=Float.parseFloat(line.split(",")[1]);//extract movement in y direction Point point = MouseInfo.getPointerInfo().getLocation(); //Get current mouse position float nowx=point.x; float nowy=point.y; robot.mouseMove((int)(nowx+movex),(int)(nowy+movey));//Move mouse pointer to new location } //if user taps on mousepad to simulate a left click else if(line.contains("left_click")){ //Simulate press and release of mouse button 1(makes sure correct button is pressed based on user's dexterity) robot.mousePress(InputEvent.BUTTON1_DOWN_MASK); robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK); } //Exit if user ends the connection else if(line.equalsIgnoreCase("exit")){ isConnected=false; //Close server and client socket server.close(); client.close(); } } catch (IOException e) { System.out.println("Read failed"); System.exit(-1); } } } }

Save the above code in RemoteDroidServer.java file and then compile, execute using the usual Java way.

javac RemoteDroidServer.java
java RemoteDroidServer

This will start the server and wait for a client to connect. After starting the server, run the android app and try to connect to your server.

NOTE:- make sure you have specified the correct server IP address in the app. Also, that the port numbers are same on both server and app.

SOURCE CODE AND MORE IDEAS

This code has been tested with a Windows 8.1 machine and a MacBook running OS X Yosemite, it should most probably work on all other operating systems as well. Server code is available above and can be further referenced from this gist and the app source is also available on github.

NOTE:- This code is only for demo purpose and should not be used as is in production.

A simple VLC remote might or might not be the thing you are looking for, but the underlying concept is very interesting. You can extend this to any level. Create a remote control for any specific app or a general purpose PC remote or even create a Game pad. Phones now a days have a lot of sensors which can be used in interesting ways, like a phone’s accelerometer and gyroscope can be used to know the tilt and control a car in any racing game. The possibilities are endless.