Master SIAME | Université Toulouse 3

Internet of things and System on Chip

Master SIAME | Université Toulouse 3

Internet of things and System on Chip

User Tools


new OpenCV & video4linux RPi camera support


Finally, it happened: as of 2014, May, the new 2.4.9 OpenCV release features support for Raspberry Pi camera module :)
This will leverage the needs for a seamless usage of our camera within openCV code.

RPi camera module.RPi and camera attached.

However, to benefits from openCV support of RPi camera, we ought to:


Warning: very long operation The whole process leading you to a new working openCV with RPi camera support will take almost 10 hours!


RPi upgrade

To benefits from the new video for linux driver that exposes an interface to openCV, you need first to fully update your RPi.

sudo su
apt-get -y update
apt-get -y upgrade
/sbin/reboot

Note: last reboot may not be usefull if there has not been any kernel upgrade.

Now we'll update the RPi's firmware itself.

sudo su
apt-get -y install rpi-update
rpi-update
/sbin/reboot

Note: You may want to restart a firmware upgrade anew after reboot occured.


Enable camera support

To enable camera support @ system-level on the RPi:

raspi-config

Then select enable camera.

As an alternative, you can also enable camera support using CLI:

cat >> /boot/config.txt

# for more options see http://elinux.org/RPi_config.txt
start_x=1
gpu_mem=128
EOF


Permanent kernel module

The most convenient way is to have Video for Linux kernel module bcm2835-v4l2 loaded at boot time:

cat >> /etc/modules << EOF
# v4l2 camera
bcm2835-v4l2
EOF

Upon next reboot, the kernel module will be loaded –> #lsmod | grep -i v4l2 to check.
As an alternative, you may also load the kernel module immediately using modprobe bcm2835-v4l2.

Below is an excerpt of the v4l2 kernel module documentation:

    # Load the module
    sudo modprobe bcm2835-v4l2

    # Control the viewfinder
    v4l2-ctl --overlay=1 # enable viewfinder
    v4l2-ctl --overlay=0 # disable viewfinder

    # Record a video
    v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=4
    v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.264

    # Capture a JPEG image
    v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
    v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=somefile.jpg

    # Set the video bitrate
    v4l2-ctl --set-ctrl video_bitrate=10000000
    
    # Set the video framerate
    v4l2-ctl --set-parm=5

    # List the supported formats
    v4l2-ctl --list-formats


openCV install

Finally, you're now ready to launch the longuest part of this tutorial: lastest openCV installation.
Some of the information below have been borrowed over the Internet and adapted to the available software at the time of writing (2014, June).

http://robertcastle.com/2014/02/installing-opencv-on-a-raspberry-pi/

[1] required packages

sudo su
apt-get -y install build-essential cmake cmake-curses-gui pkg-config libpng12-0 \
libpng12-dev libpng++-dev libpng3 libpnglite-dev zlib1g-dbg zlib1g zlib1g-dev \
pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools libeigen3-dev

apt-get -y install libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs ffmpeg \
libavcodec-dev libavcodec53 libavformat53 libavformat-dev libgstreamer0.10-0-dbg \
libgstreamer0.10-0 libgstreamer0.10-dev libxine1-ffmpeg libxine-dev libxine1-bin \
libunicap2 libunicap2-dev swig libv4l-0 libv4l-dev python-numpy python-dev  libgtk2.0-dev


[2] retrieve openCV lastest archive
At the time of writing, latest available openCV archive is rev. 2.4.9

wget http://downloads.sourceforge.net/project/opencvlibrary/opencv-unix/2.4.9/opencv-2.4.9.zip


[3] unzip and prepare to build

unzip opencv-2.4.9.zip
cd opencv-2.4.9
mkdir release
cd release
ccmake ../


[4] build options
After having launched ncurse version of the cmake gui, press c to configure with the following options:

 ANT_EXECUTABLE                   ANT_EXECUTABLE-NOTFOUND
 BUILD_DOCS                       OFF
 BUILD_EXAMPLES                   OFF
 BUILD_JASPER                     ON
 BUILD_JPEG                       ON
 BUILD_OPENEXR                    ON
 BUILD_PACKAGE                    ON
 BUILD_PERF_TESTS                 ON
 BUILD_PNG                        ON
 BUILD_SHARED_LIBS                ON
 BUILD_TBB                        OFF
 BUILD_TESTS                      ON
 BUILD_TIFF                       ON
 BUILD_WITH_DEBUG_INFO            ON
 BUILD_ZLIB                       ON
 BUILD_opencv_apps                ON
 BUILD_opencv_calib3d             ON
 BUILD_opencv_contrib             ON
 BUILD_opencv_core                ON
 BUILD_opencv_features2d          ON
 BUILD_opencv_flann               ON
 BUILD_opencv_gpu                 ON
 BUILD_opencv_highgui             ON
 BUILD_opencv_imgproc             ON
 BUILD_opencv_legacy              ON
 BUILD_opencv_ml                  ON
 BUILD_opencv_nonfree             ON
 BUILD_opencv_objdetect           ON
 BUILD_opencv_ocl                 ON
 BUILD_opencv_photo               ON
 BUILD_opencv_python              ON
 BUILD_opencv_stitching           ON
 BUILD_opencv_superres            ON
 BUILD_opencv_ts                  ON
 BUILD_opencv_video               ON
 BUILD_opencv_videostab           ON
 BUILD_opencv_world               OFF
 CLAMDBLAS_INCLUDE_DIR            CLAMDBLAS_INCLUDE_DIR-NOTFOUND
 CLAMDBLAS_ROOT_DIR               CLAMDBLAS_ROOT_DIR-NOTFOUND
 CLAMDFFT_INCLUDE_DIR             CLAMDFFT_INCLUDE_DIR-NOTFOUND
 CLAMDFFT_ROOT_DIR                CLAMDFFT_ROOT_DIR-NOTFOUND
 CMAKE_BUILD_TYPE                 Release
 CMAKE_CONFIGURATION_TYPES        Debug;Release
 CMAKE_INSTALL_PREFIX             /usr/local
 CMAKE_VERBOSE                    OFF
 CUDA_BUILD_CUBIN                 OFF
 CUDA_BUILD_EMULATION             OFF
 CUDA_HOST_COMPILER               /usr/bin/gcc
 CUDA_SDK_ROOT_DIR                CUDA_SDK_ROOT_DIR-NOTFOUND
 CUDA_SEPARABLE_COMPILATION       OFF
 CUDA_TOOLKIT_ROOT_DIR            CUDA_TOOLKIT_ROOT_DIR-NOTFOUND
 CUDA_VERBOSE_BUILD               OFF
 EIGEN_INCLUDE_PATH               /usr/include/eigen3
 ENABLE_COVERAGE                  OFF
 ENABLE_NEON                      OFF
 ENABLE_NOISY_WARNINGS            OFF
 ENABLE_OMIT_FRAME_POINTER        ON
 ENABLE_PRECOMPILED_HEADERS       ON
 ENABLE_PROFILING                 OFF
 ENABLE_SOLUTION_FOLDERS          OFF
 ENABLE_VFPV3                     OFF
 EXECUTABLE_OUTPUT_PATH           /root/opencv-2.4.9/release/bin
 GIGEAPI_INCLUDE_PATH             GIGEAPI_INCLUDE_PATH-NOTFOUND
 GIGEAPI_LIBRARIES                GIGEAPI_LIBRARIES-NOTFOUND
 INSTALL_CREATE_DISTRIB           OFF
 INSTALL_C_EXAMPLES               OFF
 INSTALL_PYTHON_EXAMPLES          OFF
 INSTALL_TESTS                    OFF
 INSTALL_TO_MANGLED_PATHS         OFF
 OPENCV_CONFIG_FILE_INCLUDE_DIR   /root/opencv-2.4.9/release
 OPENCV_EXTRA_MODULES_PATH
 OPENCV_WARNINGS_ARE_ERRORS       OFF
 OPENEXR_INCLUDE_PATH             OPENEXR_INCLUDE_PATH-NOTFOUND
 PVAPI_INCLUDE_PATH               PVAPI_INCLUDE_PATH-NOTFOUND
 PYTHON_NUMPY_INCLUDE_DIR         /usr/lib/pymodules/python2.7/numpy/core/include
 PYTHON_PACKAGES_PATH             lib/python2.7/dist-packages
 SPHINX_BUILD                     SPHINX_BUILD-NOTFOUND
 WITH_1394                        OFF
 WITH_CUBLAS                      OFF
 WITH_CUDA                        OFF
 WITH_CUFFT                       OFF
 WITH_EIGEN                       ON
 WITH_FFMPEG                      ON
 WITH_GIGEAPI                     OFF
 WITH_GSTREAMER                   ON
 WITH_GTK                         ON
 WITH_JASPER                      ON
 WITH_JPEG                        ON
 WITH_LIBV4L                      ON
 WITH_NVCUVID                     OFF
 WITH_OPENCL                      OFF
 WITH_OPENCLAMDBLAS               OFF
 WITH_OPENCLAMDFFT                OFF
 WITH_OPENEXR                     ON
 WITH_OPENGL                      ON
 WITH_OPENMP                      OFF
 WITH_OPENNI                      OFF
 WITH_PNG                         ON
 WITH_PVAPI                       ON
 WITH_QT                          OFF
 WITH_TBB                         OFF
 WITH_TIFF                        ON
 WITH_UNICAP                      OFF
 WITH_V4L                         ON
 WITH_VTK                         OFF
 WITH_XIMEA                       OFF
 WITH_XINE                        OFF

… then press 'g' to generate the various makefiles.

[5] build | the 10 hours step!
In order to ease this process, best is to launch it within a tmux session:

  apt-get -y install tmux
  cat >> ~/.tmux.conf << EOF
  # remap prefix to Control + a
  set -g prefix C-a
  unbind C-b
  bind C-a send-prefix

  # force a reload of the config file
  unbind r
  bind r source-file ~/.tmux.conf

  # quick pane cycling with C-a C-a
  unbind ^A
  bind ^A select-pane -t :.+
  EOF

Note: this tmux config files let tmux to behave the same as screen with the same command set.

We'll now launch a tmux session, launch build and disconnect from session:

tmux
make && make install

//type CTRL + a then d for disconnect//

To reconnect to a running tmux session tmux attach.

Congratulation, you've just installed latest openCV version with video4linux support for RPi camera :)

Sample code

You'll find below some simple examples making use of your freshly installed new openCV version.

opencv_test.py
#!/usr/bien/env python
# -*-coding:Utf-8-*
import cv2.cv as cv
 
#default to capture from /dev/video0
capture = cv.CaptureFromCAM(0)
while True:
   # Grab one frame from camera
   frame = cv.QueryFrame(capture)
 
   # Display frame in a window
   cv.ShowImage( "image", frame )
 
   # Wait for ESC to end program
   c = cv.WaitKey(7) % 0x100
   if c == 27:
      break
cv.DestroyAllWindows()

… then launch this application test

chmod a+x opencv_test.py
./opencv_test.py

Note: ESC key to quit.

Another test using this time the openCV's C libraries.

opencv_test.cpp
#include "opencv2/highgui/highgui.hpp"                                             
#include <iostream>                                                                
 
using namespace std;
using namespace cv;
 
int main(int argc, char** argv){
    VideoCapture cap(-1);
    if (!cap.isOpened())
    {
        cout << "Cannot open camera" << endl;
        return -1;
    }
   cap.set(CV_CAP_PROP_FRAME_WIDTH, 640);
   cap.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
 
   namedWindow("Output",CV_WINDOW_AUTOSIZE);
 
    while (1)
    {
        Mat frame;
        bool bSuccess = cap.read(frame);
 
        if (!bSuccess)
        {
        cout << "Cannot read a frame from camera" << endl;
        break;
        }
        imshow("Output", frame);
 
        if (waitKey(30) == 27)
        {
        cout << "Exit" << endl;
        break;
        }
    }
    return 0;
}

Code compilation and launch

gcc opencv_test.cpp -o opencv_test -lopencv_core -lopencv_highgui
./opencv_test

As an alternative, you can check the availability of the libs @ system.
pkg-config: to test availability of the opencv package @ system level

pkg-config --libs opencv
/usr/local/lib/libopencv_calib3d.so /usr/local/lib/libopencv_contrib.so /usr/local/lib/libopencv_core.so
/usr/local/lib/libopencv_features2d.so /usr/local/lib/libopencv_flann.so /usr/local/lib/libopencv_gpu.so
/usr/local/lib/libopencv_highgui.so /usr/local/lib/libopencv_imgproc.so /usr/local/lib/libopencv_legacy.so
/usr/local/lib/libopencv_ml.so /usr/local/lib/libopencv_nonfree.so /usr/local/lib/libopencv_objdetect.so
/usr/local/lib/libopencv_photo.so /usr/local/lib/libopencv_stitching.so /usr/local/lib/libopencv_superres.so
/usr/local/lib/libopencv_ts.a /usr/local/lib/libopencv_video.so /usr/local/lib/libopencv_videostab.so -lrt -lpthread -lm -ldl                                                                                         


openCV build@SL65

To install openCV @ Scientific Linux 6.5, the same procedure applies as for the RPi version except libs dependencies resumed below:

[Nov. 14]: updated to OpenCV 3.0

yum -y install cmake28 cmake28-gui gcc gcc-c++ gtk2-devel gtk+-devel libpng-devel libjpeg-devel libtiff-devel jasper-devel libpng-devel zlib-devel eigen3-devel ffmpeg ffmpeg-devel bzip2-devel

Note: no rtsp support without ffmpeg!

… and do not forget to add libs paths to the LD_LIBRARY_PATH env var:

cat >> /etc/ld.so.conf.d/opencv.conf << EOF
/usr/local/lib
EOF

… and activate the new paths:

ldconfig


openCV build in a VM

Warning: qemu-kvm illegal instruction issue When building openCV in a VM, you must pay attention to processor's features


#> cat /proc/cpuinfo | grep -E "^flags"
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16 hypervisor lahf_lm

Here are typical processor flags of a qemu-kvm VM instance. This leads to some adjustements to the openCV compilation flags:

......
ENABLE_AVX                 OFF
ENABLE_SSE3                OFF
ENABLE_SSE4                OFF
ENABLE_SSE4_1              OFF
ENABLE_SSE4_2              OFF
ENABLE_SSSE3               OFF
......