How does WebADB work (part 3) - Stream

Last time we have successfully connected to devices. Now we can try to send some commands.

ADB commands are a lot like HTTP requests.

  • Each request has a url called “service string” to specify it’s type.
  • Simpler requests embed parameters in “service string”, like GET requests.
  • More complex requests send extra data to the stream before daemon responses, just like POST requests.

But actually ADB streams are fully duplex, client and daemon can send data to each other at any time.

Let me explain it in more detail.

Control Packet

In last post we seen two types of ADB packets: CNXN for connecting and AUTH for authorization.

There are some other packets for stream multiplexing:

OPEN packet

An OPEN packet requests the other side to create a new stream. Most time streams are initiated by clients, the only exception is the “reverse forwarding” feature where the daemon initiates a stream.

Field Value Description
command 0x4e45504f ‘OPEN’ in UTF-8 encoding
arg0 local id Starting from 1 and auto increasing
arg1 0 not used
payload {destination}:{parameters} Service string

The service string is a little bit massy

  1. Android <9 requires an extra null character at end
  2. Every commands’ format is slightly different, so I will explain more in each commands’ section

Note: “local id” on the transmitting side is actually “remote id” on the receiving side, and vice versa.

CLSE packet

If the other end don’t want to create the stream, for example because the service string was invalid, it will send a CLSE packet.

Field Value Description
command 0x45534c43 ‘CLSE’ in UTF-8 encoding
arg0 0 0 means OPEN stream failed
arg1 remote id Same as arg0 of the OPEN packet
payload EMPTY

OKAY packet

If the other end successfully created the stream, it sends back an OKAY packet.

Field Value Description
command 0x59414b4f ‘OKAY’ in UTF-8 encoding
arg0 local id Starting from 1 and auto increasing
arg1 remote id Same as arg0 of the OPEN packet
payload EMPTY

For example, the client want to open a stream with service string usb: (for switching the daemon to usb mode), so far it should look like this:

Not the client can find the stream with local id 1 and set its state to connected.

WITE packet

Field Value Description
command 0x45534c43 ‘WITE’ in UTF-8 encoding
arg0 local id
arg1 remote id
payload data

The WITE packet writes data to the stream so it can be read from other side. I have already said that each side can write data to stream at any time.

The payload’s size must not exceed the max payload size negotiated between client and daemon on connection. However, multiple WRIT packets can be used to send a long message.

The receiving end will send back a OKAY packet to indicate that it has received the data. The transmitting side must not send another WITE before receiving a OKAY packet.

For example, the client has already created a stream, which has local id 10 and remote id 20, now it want to send 500 bytes of data to it, however the max payload size is only 300 bytes.

Another example, the client and daemon both want to send data over the same stream:

Close a stream

Both sides can close a stream, no matter which side initiated it, at any time, by sending a CLSE packet with a non-zero arg0 (local id).

Blocking

Although it looks like there are multiple streams, in fact they all transfer through a single connection. So if you don’t read the available data from stream 1, it will block data on all other streams.

To solve this problem, each stream can have its own buffer, or multiple threads can be created to receive and process data.

Implementation

Dispatcher

To implement the stream multiplexer, I created a new class call AdbPacketDispatcher: In fact, not only stream control packets, it’s responsible for reading all packets, including the CNXN and AUTH packets we talked earlier.

The point is that there is only one location in the whole package to read packets:

  1. If the reading logic ever requires changes, for example the format is changed, or I want to add some logging, I only need to change this one location.
  2. If some new packet types was added, and new code is needed to handle them, I don’t need to duplicate the reading logic again, I can just hook it to the dispatcher.

The dispatcher only knows how to handle stream control packets, but it has an onPacket event so other modules can attach a listener for any packets they interested.

Stream

The AdbStream class itself is event-driven, an onData event and an onClose event corresponding to the WITE and CLSE packets received. It’s the most simple form of implementation.

However many times it’s more convenient to use a pulling pattern (calling some function and wait it to return some value). So I first created an EventQueue helper class that can transform any event into a pulling system and supports buffering.

Then the AdbReadableStream class was added. It uses the EventQueue‘s buffer to mitigate blocking issue, and it provides a read method that returns a Promise which will eventually resolves to newly received data.

But it still has an issue. AdbReadableStream‘s read method always returns as much data as the operating systems returns, sometimes other parts may only need exactly N bytes. So I created another BufferedStream helper class, it can take anything that returns ArrayBuffers, split and rearrange them into the requested size. It’s used by the AdbBufferedStream class, that wraps an AdbReadableStream.

Looks complex? In fact it’s pretty clear:

  1. Each pattern has its usage. For example pulling pattern requires a reading loop while event pattern doesn’t.
  2. Each class has only one way to consume. If all three patterns are in the same class, I can’t even imagine what will happen if I use the event and the read method at the same time.
  3. Each helper class is generic and can be used by other similar requirement. Actually I also used the EventQueue in the demo project to convert another event into pulling pattern.

Commands

Commands are identified by the service string. Most commands are sent by clients.

Let’s see some simple commands:

usb:

This command turns off TCP port listening on the daemon. It doesn’t have any parameter in the service string, and client doesn’t send any data over the opened stream.

(After the OKAY packet to confirm the stream is opened) the server will send restarting in USB mode\n over the stream, then CLSE it, and restart itself (dropping the connection).

tcpip:{port}

This command tells the daemon to listen on the specified port. {port} is a UTF-8 encoded number that >0 and <65536.

Like usb:, the client doesn’t transfer anything over the stream. The daemon will respond restarting in TCP mode port: {port}\n, CLSE the stream, and restart.

What’s next

Next time is about the sync: command. It’s more complicated: both sides will transfer more data over the opened stream to achieve file operations, including listing, downloading and uploading.