Real-Time Tracking of Multiple Moving Mosquitoes

Lighting, Camera, and Lens Setup

• Proper alignment/setup of all cameras is important from experimental inception (see Protocol: Designing a Generic Videography Experiment for Studying Mosquito Behavior [Muijres et al. 2022]). When the camera orientation, zoom, aperture, or focus position is changed, one must reconsider the camera calibration for both lens distortions and stereo calibration of the multiple cameras in the tracking system. The pylon Viewer provided by Basler (2021) provides some useful features for camera alignment such as a grid, histogram, and focus assistant.

• 1. Launch pylon Viewer.

• 2. Connect all cameras and start the video stream for each. Set camera trigger mode to “off” and use free run.

• 3. Arrange infrared lights such that the entire tracking volume is well lit for all cameras.

• 4. Fully open the lens aperture.

• 5. Adjust the exposure time to get a well-lit scene and so that the reference points in the setup are clearly visible; adjust the focus if needed. Frame rate is not yet important for this step.

  • Use the camera's histogram to check whether the entire dynamic range is used. Sensor gain or pixel binning is optional to increase image brightness.

• 6. Adjust the camera orientation and lens zoom such that as much of the tracking volume (or part of it) is in view.

  • Stop down the lens aperture to increase the depth of field.

• 7. Repeat Steps 1–6 for each camera. Ensure that each point in the tracking volume is visible by at least two camera views. A 90° angle between cameras maximizes stereo information. More cameras are helpful to improve tracking performance.

Synchronization Cables and Trigger Box Setup

• 8. Connect each camera with the trigger cable to the Arduino Nano.

Configuring Braid to Connect to Your Cameras

• 9. Prior to running Braid, generate a configuration file with the identity of the cameras, a calibration, if available, and tracking parameters. Any values not explicitly set in the configuration file will default to reasonable values.

• 10. To see all possible configuration items and their default settings, type “braid default-config” at the command line. Save the output of this program to a TOML file like “config.toml” then edit the file name “config.toml” to include the individual names of your cameras. Remove all values for which the defaults are OK.

• 11. Ensure your configuration is correct by running “braid show-config config.toml.”

Camera Intrinsic Parameter Calibration

• 12. Start Braid with “braid run config.toml.”

  • Two stages of camera calibration are required, and they must be performed in sequential order. The first stage calibrates the intrinsic parameters of the camera. These are the focal length, optical center pixel, and any image distortion but do not include the position of the camera.

• 13. From the Braid main page, launch Strand Cam for the first camera to open the Strand Cam user interface for each camera.

• 14. Disable object detection by unchecking the “object detection” checkbox and go to the “checkerboard calibration” panel.

• 15. Enter the checkerboard size in the user interface.

• 16. Click “enable checkerboard calibration.”

• 17. Collect at least 10 checkerboard images by holding the checkerboard in front of the camera. To accurately calibrate the camera's distortion, which is highest near the corners, ensure the checkerboard is visible in the corners of the image.

• 18. To perform and save the calibration, click the “perform and save calibration” button. Repeat this procedure for all cameras.

Multicamera Calibration

• Here, the MultiCamSelfCal calibration routine (Svoboda et al. 2005) is used to create a single calibration for all cameras.

• 19. Reduce camera exposure duration and set the object detection parameters in Strand Camera so that the LED is detected when turned on, but no other points are detected.

  • Turn off the infrared lights used for illumination.

• 20. Begin recording a data file in the Braid user interface.

• 21. Wave the LED smoothly through the tracking volume and ensure it tracked from all cameras, with at least some frames where it is visible from three or more cameras.

  • Turn the LED on and off intermittently while moving through the tracking volume to validate synchronization.

• 22. Stop saving the data.

• 23. Run the MultiCamSelfCal calibration code to generate an initial multicamera calibration that is not yet “aligned.”

• 24. Align the new calibration to the correct coordinate frame with the “flydra analysis calibration align GUI” tool.

  • Use an LED to track a path along a known path in the coordinate frame.

Data Collection

• 25. In the Braid user interface, begin recording a data file.

• 26. Perform your experiment.

• 27. Stop saving the data. Braidz files can be viewed using the viewer at https://braidz.strawlab.org/ and analyzed starting with example scripts available in the User's Guide for Braid and Strand Camera (Straw et al. 2011; Straw 2021).