RICOH THETA Live Preview

Craig Oda

Jul 21, 2022 • 2 min read

The live preview from the RICOH THETA camera is accessed with the camera.getLivePreview API command.

Receiving Stream Instead of JSON

Unlike other API commands such as camera.takePicture, the getLivePreview command will return a stream. Your app is likely set up to receive JSON from the RICOH THETA camera. When you send the camera.getLivePreview command to THETA, you should prepare your application to receive a stream.

The thetax_live_preview demonstration from Oppkey uses the responseType property from dio. Valid values are json, stream, and plain. Specify ResponseType.stream.

import 'package:dio/dio.dart';
import 'package:theta/theta.dart';

const Map<String, dynamic> emptyBody = {};

/// use ResponseType.stream if using camera.getLivePreview
/// and the response is a stream
Future<dynamic> command(String baseName,
    {Map<String, dynamic> parameters = emptyBody,
    ResponseType responseType = ResponseType.stream,
    additionalHeaders = emptyBody}) async {
  var response = await ThetaBase.post('commands/execute',
      responseType: responseType,
      additionalHeaders: additionalHeaders,
      body: {
        'name': 'camera.$baseName',
        'parameters': parameters,
      });

  return response;
}

I am using this as the header:

    Map<String, dynamic> additionalHeaders = {
      'Accept': 'multipart/x-mixed-replace'
    };

Demonstration of the code in action with overview of MotionJPEG

Listen To Stream

Once your application is set up to receive a stream, it must listen to the stream so that you can parse the chunks and look for the start and stop characters for each JPEG frame.

    subscription = dataStream.listen((chunkOfStream) {
      if (frameCount > frames && frames != -1 && keepRunning) {
        if (subscription != null) {
          subscription.cancel();
          controller.close();
        }
      }

Identify the Beginning and End of Each JPEG Frame

The start of each frame is 0xd8. The end of each frame is 0xd9. While your application is listening to the stream, it will receive an iterable, chunkOfStream.

You can use a simple for loop to iterate through each byte of the stream. If the byte is 0xd8, save the index of the chunkOfStream into a temporary variable.

Repeat the process with the end byte of 0xd9.

Here's the code snippet to save the frames into a buffer. The buffer is a simple list of integers List<int> buffer = []

Each element of the list will be a JPEG image encoded as bytes.

      if (keepRunning) {
        buffer.addAll(chunkOfStream);
        // print('current chunk of stream is ${chunkOfStream.length} bytes long');

        for (var i = 1; i < chunkOfStream.length; i++) {
          if (chunkOfStream[i - 1] == 0xff && chunkOfStream[i] == 0xd8) {
            startIndex = i - 1;
          }
          if (chunkOfStream[i - 1] == 0xff && chunkOfStream[i] == 0xd9) {
            endIndex = buffer.length;
          }

          if (startIndex != -1 && endIndex != -1) {
            var frame = buffer.sublist(startIndex, endIndex);
            if (frameTimer.elapsedMilliseconds > frameDelay) {
              if (frameCount > 0) {
                controller.add(frame);

                print('framecount $frameCount');
                frameTimer.reset();
              }

              frameCount++;
            }
            // print(frame);
            startIndex = -1;
            endIndex = -1;
            buffer = [];
          }
        }
      } //

RICOH THETA X test of this project with iPhone

Display JPEG Images

Flutter can display images to the screen using Image.memory. With this technique, I also needed to convert the list into a Uint8List. Here's the code to display the images to the screen as motionJPEG

  @override
  Widget build(BuildContext context) {
    return StreamBuilder(
      stream: widget.controller.stream,
      builder: (BuildContext context, AsyncSnapshot snapshot) {
        if (snapshot.hasData) {
          var imageData = Uint8List.fromList(snapshot.data);
          return Image.memory(
            imageData,
            gaplessPlayback: true,
          );
        } else {
          return Container();
        }
      },
    );
  }