The CMUcam5 Pixy camera is available in a version designed for LEGO Mindstorms. In this tutorial we explain how to use it with ev3dev and python.
General information about Pixy for LEGO Mindstorms can be found on Charmed Lab’s wiki-page.
The Pixy camera comes with its own tool: PixyMon. This tool helps you to set the signatures (objects you want Pixy to detect). To use this tool you need to connect the camera directly to your PC by using a mini USB cable. For this you can use the USB cable of your EV3. Beware that when PixyMon is running on your PC (and the camera is plugged in to your PC), its values are not updated to the EV3. So before starting your script be sure PixyMon is not running!
In python use the Sensor class to connect to the Pixy camera:
from ev3dev.ev3 import *
pixy = Sensor(address=INPUT_1)
assert pixy.connected, "Error while connecting Pixy camera to port1"
In this statement it is assumed the camera is connected to input port 1.
It’s recommended to use the assert statement: when the connection fails
the program ends with a meaningful error message.
Next set the mode for the camera:
pixy.mode = 'ALL'
The Pixy camera has the following modes:
The data which you retrieve from the camera depends on the camera mode. You can find detailed information on this page. We will explain it to you with some examples.
When the mode is set to ALL, you can retrieve data as follows:
sig = pixy.value(1)*256 + pixy.value(0) # Signature of largest object
x_centroid = pixy.value(2) # X-centroid of largest SIG1-object
y_centroid = pixy.value(3) # Y-centroid of largest SIG1-object
width = pixy.value(4) # Width of the largest SIG1-object
height = pixy.value(5) # Height of the largest SIG1-object
When mode is set to one of the signatures (e.g. SIG1), retrieve data
as follows:
count = pixy.value(0) # The number of objects that match signature 1
x = pixy.value(1) # X-centroid of the largest SIG1-object
y = pixy.value(2) # Y-centroid of the largest SIG1-object
w = pixy.value(3) # Width of the largest SIG1-object
h = pixy.value(4) # Height of the largest SIG1-object
Below are two practical examples. They are tested with the latest version of ev3dev (as of 21 Dec 2016) and with CMUcam5 Pixy camera for LEGO Mindstorms.
To run the scripts form Brickman, be sure it’s executable. To make it
executable run chmod +x <file> once in the terminal. Also don’t forget
the shebang #!/usr/bin/env python3 in the first line of your script.
This example uses an EV3 with a Pixy camera and a TouchSenor plugged in. In this example the camera searches for objects with signature SIG1 and displays the data graphically on the LCD of the EV3. It draws a rectangle centered at the X and Y centroid coordinates with measured width and height. End this script by pressing the TouchSensor.
When running this script from the SSH terminal, the Brickman display may reappear at any time, momentarily or permanently. Furthermore, when the script stops it may take a while to before the screen reverts back to the Brickman interface. To prevent this, follow these instructions:
sudo chvt 6 before starting the script. This will clear the display.
For this command you need the password for robot, which is ‘maker’ by default.sudo chvt 1 to bring back the interface of Brickman.#!/usr/bin/env python3
from ev3dev.ev3 import *
lcd = Screen()
# Connect Pixy camera
pixy = Sensor(address=INPUT_1)
assert pixy.connected, "Connecting PixyCam"
# Connect TouchSensor
ts = TouchSensor(address=INPUT_4)
assert ts.connected, "Connecting TouchSensor"
# Set mode
pixy.mode = 'SIG1'
while not ts.value():
lcd.clear()
if pixy.value(0) != 0: # Object with SIG1 detected
x = pixy.value(1)
y = pixy.value(2)
w = pixy.value(3)
h = pixy.value(4)
dx = int(w/2) # Half of the width of the rectangle
dy = int(h/2) # Half of the height of the rectangle
xb = x + int(w/2) # X-coordinate of bottom-right corner
yb = y - int(h/2) # Y-coordinate of the bottom-right corner
lcd.draw.rectangle((xa,ya,xb,yb), fill='black')
lcd.update()
This video shows this script running:
This example uses an EV3 with Pixy camera, two LargeMotors and a TouchSensor. The robot follows the object with signature SIG1. To stop the program the user has to press the TouchSensor.
This example uses a simple implementation for a PID-controller. The example works very well, but fine tuning of the PID-constants can make the robot react and move smoother.
#!/usr/bin/env python3
from ev3dev.ev3 import *
def limit_speed(speed):
""" Limit speed in range [-900,900] """
if speed > 900:
speed = 900
elif speed < -900:
speed = -900
return speed
# Connect Pixy camera and set mode
pixy = Sensor(address = INPUT_1)
assert pixy.connected, "Error while connecting Pixy camera"
pixy.mode = 'SIG1'
# Connect TouchSensor (to stop script)
ts = TouchSensor(INPUT_4)
assert ts.connected, "Error while connecting TouchSensor"
# Connect LargeMotors
rmotor = LargeMotor(OUTPUT_A)
assert rmotor.connected, "Error while connecting right motor"
lmotor = LargeMotor(OUTPUT_D)
assert lmotor.connected, "Error while connecting left motor"
# Defining constants
X_REF = 128 # X-coordinate of referencepoint
Y_REF = 150 # Y-coordinate of referencepoint
KP = 0.4 # Proportional constant PID-controller
KI = 0.01 # Integral constant PID-controller
KD = 0.005 # Derivative constant PID-controller
GAIN = 10 # Gain for motorspeed
# Initializing PID variables
integral_x = 0
derivative_x = 0
last_dx = 0
integral_y = 0
derivative_y = 0
last_dy = 0
while not ts.value():
if pixy.value(0) > 0:
x = pixy.value(1) # X-centroid of largest SIG1-object
y = pixy.value(2) # Y-centroid of largest SIG1-object
dx = X_REF - x # Error in reference to X_REF
integral_x = integral_x + dx # Calculate integral for PID
derivative_x = dx - last_dx # Calculate derivative for PID
speed_x = KP*dx + KI*integral_x + KD*derivative_x # Speed in X-direction
dy = Y_REF - y # Error in reference to Y_REF
integral_y = integral_y + dy # Calculate integral for PID
derivative_y = dy - last_dy # Calculate derivative for PID
speed_y = KP*dy + KI*integral_y + KD*derivative_y # Speed in Y-direction
# Calculate motorspeed out of speed_x and speed_y
# Use GAIN otherwise speed will be to slow, but limit in range [-900,900]
rmotor.run_forever(speed_sp = limit_speed(GAIN*(speed_y + speed_x)))
lmotor.run_forever(speed_sp = limit_speed(GAIN*(speed_y - speed_x)))
last_dx = dx # Set last error for x
last_dy = dy # Set last error for y
else:
rmotor.stop() # SIG1 not detected, stop motors
lmotor.stop()
# End of script, stop motors
rmotor.stop()
lmotor.stop()
This video shows this script running:
.