What is Pixy2?The Pixy2 is a tiny camera designed primarily for the following three functions:
- Object detection (road signs, intersections, etc.)
- Line detection and tracking
- Barcode detection capabilities/Barcode reading capabilities
The Pixy2 CameraSource: Flickr
Pixy2 Camera Features and Specifications
- Compact, quick, affordable, easy-to-use, and readily available vision system
- Detects and tracks intersections, lines, and tiny barcodes for line-following robots
- 60 FPS
- Color-based object detection using tracking algorithms
- Built-in light source
- Improved, simplified libraries for Raspberry Pi, Arduino, and other single board computer controllers
- Supports python and C/C++ libraries of code
- A multitude of interface options (I2C, SPI, USB, UART, and analog/digital output)
- Windows, Linux, and macOS compatible configuration utility
- Connects to the controllers via the included cable
- Dual-core, 204 MHz, NXP LPC4330 processor
- 1296×976 resolution Aptina MT9M114 dual-core image sensor
- 60° horizontal, 40° vertical field of view (lens)
- 140mA power consumption
- Unregulated 6-10V or 5V USB power input
- 264KB RAM & 2MB flash memory
- USB, UART serial, I2C, SPI, digital, and analog output ports
- Built-in 20-lumen light source
The Pixy2 CameraSource: Flickr
Getting Started Pixy2Before using the Pixy2, you should install it on your computer using the following steps.
Install PixyMonThe installation requires a high-quality micro USB cable and the latest PixyMon version.
Mac InstallationDouble click the downloaded DMG file to mount it. A PixyMon application window will pop up. Drag the app to your applications directory, and that's it for software installation. Next, connect the Pixy2 camera to your computer using the USB cable. Double click to launch the installed app, and it will connect to the camera.
Windows Vista, 7, 8, and 10 InstallationAfter downloading the latest Windows PixyMon version, double click to install. Proceed with the installation until you get to the "driver pre-installation" part. After a few moments, you should get a dialog box with the text "Would you like to install this device software?" Click "install." The same dialog box will pop up again after a while because the camera has two device drivers that need installation. Click "install" again, and once done, connect the Pixy2 to your computer using the USB cable and run PixyMon.
Linux InstallationThis process assumes you are running Linux Mint (or any Debian package manager) and connected to the internet.
Dependency InstallationNext, install the PixyMon dependencies by first opening the terminal. Type the following command to install git. After that, install libusb-1.0-0-dev using this command. Next, install QT by typing these three commands. The g++ compiler installation is next. Type: At this point, dependency installation is complete. The next step is building PixyMon.
PixyMon BuildingFirst, download the source code using the following command. Second, run the build script by typing: This script might take some time. Lastly, add permissions for the USB interface by running the following script.
Running PixyMonConnect the camera to your PC using the micro USB cable, then change the directory to the PixyMon build folder by typing the following command. Complete the process by running PixyMon using this command. The output should be a live video feed from the camera.
Teach Pixy2 An ObjectThe Pixy2 sensor uses a hue/color-based filtering algorithm for image detection. Therefore, the colored objects for training should have a distinct hue and include items like green tennis balls, pink dolls, etc.
Green tennis ballNext, power the camera via USB or batteries. Once all the lights have stopped flashing, press and hold the button at the top of the Pixy2. The LEDs will turn on again in the following sequence: white followed by red, then other colors. Release the button when the LED turns red. The camera will go into a light pipe mode and lock onto objects right in the middle of its video frame. Therefore, you should put the item 6-20 inches ahead of the lens. Pixy2 will determine which pixels cover the object and which ones are part of the background. The LED color should match the object's color, and the brighter the LED, the better the lock. Once satisfied with the results, press and release the Pixy 2 button. The LED will flash several times to show it has learned your object. The camera can learn up to seven distinct objects based on the color signatures. These include orange, red, yellow, cyan, violet, blue, and green.
Lighting Conditions and White BalanceIt is crucial to note that the lighting condition (such as fluorescent and incandescent lighting) can affect the white balance. This issue will affect the camera's ability to detect objects accurately. You can solve this problem by retraining the Pixy2 signatures or adjusting the white balance.
Hook Up to your Microcontroller (Arduino and Raspberry Pi)
ArduinoFirst, download the latest Arduino library and import it into the IDE to set up the connection. After that, load the example "ccc_hello_world" (File > Examples > Pixy2), upload it to the board and open the serial monitor. Also, set the baud rate to 115200. Some messages will get printed on the monitor to show what the color-connected components program has identified. However, it will only print messages if an object matching one of the CCC's color signatures is in front of the camera.
A custom made line following robot with an Arduino boardSource: Wikimedia Commons
Raspberry PiConnecting a Raspberry Pi to a Pixy2 requires the libpixyusb2 library and a PC running on Linux Mint or any Debian package manager. First, install the dependencies for building the library. Start the terminal, then type the following command. Next, download the Pixy2 source code using this command. After that, build the library using: Connect the Pixy2 to your computer, then try out one of the example projects. For instance, build the color identification example using this command. After that, start the example by running the following commands.
Pixy2 Modes / Programs
Color Connected ComponentsPixy2's color-based filtering algorithm for detecting object signatures is the Color Connected Components (CCC). Although this algorithm remembers and detects up to seven different color signatures, it can find objects in the hundreds at a time. Pixy2 compiles the location and size of each object, then outputs this data via one of the interfaces. This multiple object detection is possible via color-coding. Color codes are two or more tags placed near each other. The camera can detect and decode color codes, then output them as special objects.
Line TrackingPixy2 uses an image array sensor to follow a line, which takes in data about the line direction and intersections formed by it. The camera can also read simple barcodes to inform the robot to slow down, turn left or right, etc. However, the advanced line-tracking algorithm can only detect light lines on a dark background or vice versa. The built-in light source kicks in to make the camera work optimally if it is too dark. Also, you can program the Pixy2 to detect specific line widths only.
A line tracking robotSource: Wikimedia Commons
Pixy2 Application Projects
Introducing Pixy2 for Lego Mindstorms EV3Pixy2 communicates with the Lego sensor protocol seamlessly. You only have to plug the included cable into your lego brick, then observe the detected objects on the EV3. Alternatively, you can take the data to your Lego program using Pixy's software blocks. This software simplifies Lego program writing to drag and drop programming, with new blocks including RGB color readout, line-following, and CCC.
Lego Mindstorms EV3 robot