Setup
The following commands can be used to install v4l2 and v4l-utils.$ sudo apt update $ sudo apt install fswebcam libv4l-dev v4l-utils ffmpeg
Another utility 'fswebcam' is an application to use webcam [3]. You might need to add user to video group not to get permission denied problem.
$ sudo usermod -a -G video
Then, you can use the following command to capture an image using webcam.
$ fswebcam image.jpg
You can create a configuration to use with fswebcam. For example, create 'webcam.conf' as follows.
device /dev/video0 input 0 resolution 640x480 png 0 save img.png
Thereafter, use the following command.
$ fswebcam -c webcam.conf
You can use gpicview to view the image.
$ gpicview img.png
Another option is to use gstreamer [4].
$ sudo apt install libgstreamer1.0-0 gstreamer1.0-plugins-base \ gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly \ gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x \ gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 \ gstreamer1.0-pulseaudio $ sudo apt install gstreamer1.0-plugins-base-apps
Then use the following command to view the image [5].
$ gst-play-1.0 img.png
Another application to test capturing using webcam is 'cheese' which can be install and use as follows.
sudo apt install cheese cheese
Capture Example
Let us get 'capture.c' example fromhttps://www.kernel.org/doc/html/v4.9/media/uapi/v4l/capture-example.html
and run. You can use the following command to compile it [6].
$ gcc -O2 -Wall `pkg-config --cflags --libs libv4l2` capture.c -o capture
After compiling, you can see usage information.
$ ./capture -h
For example, to capture an image from device 0 and save it to output.raw, you can use the following option.
$ ./capture -d /dev/video0 -o -c 1 > output.raw
You can force format to 640x480 YUYV by using -f flag. Another approach to change format is using v4l2-ctl utility. Some example usage to list, and control are shown below.
$ v4l2-ctl --list-devices $ v4l2-ctl --list-formats-ext -d 0 $ v4l2-ctl --all -d 0 $ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=0,field=none -d 0
Analysis
To better understand the capture example, let us rewrite capture.c in our own way. Create a new file 'ce_capture.c' which can be found at:https://github.com/yan9a/rpi/blob/master/opencv/ce_capture.c
For simplicity, I remove getting command line options and use only MMAP for io method. The resulting main is shown below.
int main(int argc, char **argv)
{
dev_name = "/dev/video0";
open_device();
init_device();
start_capturing();
mainloop();
stop_capturing();
uninit_device();
close_device();
fprintf(stderr, "\n");
return 0;
}
Opening and Closing
Let us first look at opening and closeing device. The device can be opened asfd = open(dev_name, O_RDWR | O_NONBLOCK, 0);
We can use system call 'stat' to check device attributes before opening. For that, 'sys/stat.h' needs to be included. If the device is not a char device, we will exit the program. And we also exit the program if there is error in opening. The resulting function is shown below.
static void open_device(void)
{
struct stat st;
if (-1 == stat(dev_name, &st)) {
fprintf(stderr, "Cannot identify '%s': %d, %s\n",
dev_name, errno, strerror(errno));
exit(EXIT_FAILURE);
}
if (!S_ISCHR(st.st_mode)) {
fprintf(stderr, "%s is no device\n", dev_name);
exit(EXIT_FAILURE);
}
fd = open(dev_name, O_RDWR /* required */ | O_NONBLOCK, 0);
if (-1 == fd) {
fprintf(stderr, "Cannot open '%s': %d, %s\n",
dev_name, errno, strerror(errno));
exit(EXIT_FAILURE);
}
}
Closing the device is duely done as shown below.
static void close_device(void)
{
if (-1 == close(fd)) errno_exit("close");
fd = -1;
}
Initialization and Uninitialization
Secondly, let us look at initializing and uninitialization of the device. We use ioctl to control device. Function 'xioctl' is defined to retry to call ioctl if the cause of error is interrupt.
static int xioctl(int fh, int request, void *arg)
{
int r;
do {
r = ioctl(fh, request, arg);
} while (-1 == r && EINTR == errno);
return r;
}
When initializing the device, we try to check capabilities of the device and exit if the device is not a supported device. Querying the device is done by calling ioctl with VIDIOC_QUERYCAP as an input.
/* Check capabilities */ if (-1 == xioctl(fd, VIDIOC_QUERYCAP, &cap)) ...
After querying, its capabilities are checked with bit values V4L2_CAP_VIDEO_CAPTURE, and V4L2_CAP_STREAMING.
if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) ... if (!(cap.capabilities & V4L2_CAP_STREAMING)) ...
Then, we set the format of the video. For that, we first get the video cropping and scaling abilities by calling ioctl with VIDIOC_CROPCAP.
/* Select video input, video standard and tune here. */ CLEAR(cropcap); cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; if (0 == xioctl(fd, VIDIOC_CROPCAP, &cropcap)) ...
Then set it with VIDIOC_S_CROP call to default by setting c (cropping rectangle) value of v4l2_cropcap structure with cropcap.defrect.
crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; crop.c = cropcap.defrect; /* reset to default */ if (-1 == xioctl(fd, VIDIOC_S_CROP, &crop)) ...
Thereafter, we set the video format e.g. width, height, etc. with VIDIOC_S_FMT call.
CLEAR(fmt);
/* Set format */
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.width = 640;
fmt.fmt.pix.height = 480;
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
if (-1 == xioctl(fd, VIDIOC_S_FMT, &fmt))
errno_exit("VIDIOC_S_FMT");
The resulting initialization function is shown below which calls a function for memory mapping at the end.
static void init_device(void)
{
struct v4l2_capability cap;
struct v4l2_cropcap cropcap;
struct v4l2_crop crop;
struct v4l2_format fmt;
/* Check capabilities */
if (-1 == xioctl(fd, VIDIOC_QUERYCAP, &cap)) {
if (EINVAL == errno) {
fprintf(stderr, "%s is no V4L2 device\n",
dev_name);
exit(EXIT_FAILURE);
} else {
errno_exit("VIDIOC_QUERYCAP");
}
}
if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {
fprintf(stderr, "%s is no video capture device\n",
dev_name);
exit(EXIT_FAILURE);
}
if (!(cap.capabilities & V4L2_CAP_STREAMING)) {
fprintf(stderr, "%s does not support streaming i/o\n",
dev_name);
exit(EXIT_FAILURE);
}
/* Select video input, video standard and tune here. */
CLEAR(cropcap);
cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
if (0 == xioctl(fd, VIDIOC_CROPCAP, &cropcap)) {
crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
crop.c = cropcap.defrect; /* reset to default */
if (-1 == xioctl(fd, VIDIOC_S_CROP, &crop)) {
switch (errno) {
case EINVAL:
/* Cropping not supported. */
break;
default:
/* Errors ignored. */
break;
}
}
} else {
/* Errors ignored. */
}
CLEAR(fmt);
/* Set format */
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.width = 640;
fmt.fmt.pix.height = 480;
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
if (-1 == xioctl(fd, VIDIOC_S_FMT, &fmt))
errno_exit("VIDIOC_S_FMT");
init_mmap();
}
As we will be using memory map buffer, the function 'init_mmap' function is called in initialization procedure to perform memory mapping for it. Initialization of memory map consists of 2 parts - requesting and memory mapping. It requires at least 2 buffers for streaming and we request 4 in our case. When mapping, we query buffer information by providing buffer type and then use its length and offset in mmap.
Requesting buffer type and count is done by the following lines of code.
/* REQUEST */ struct v4l2_requestbuffers req; CLEAR(req); /* Request buffer type and count */ req.count = 4; req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; req.memory = V4L2_MEMORY_MMAP; if (-1 == xioctl(fd, VIDIOC_REQBUFS, &req)) ...
Thereafter, we create structures to access the requested buffers from our application. Query the length and offset of requested buffers with VIDIOC_QUERYBUF. And then map the memory and initialize the buffer structures. The resulting init_mmap function is shown below.
static void init_mmap(void)
{
/* REQUEST */
struct v4l2_requestbuffers req;
CLEAR(req);
/* Request buffer type and count */
req.count = 4;
req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
req.memory = V4L2_MEMORY_MMAP;
if (-1 == xioctl(fd, VIDIOC_REQBUFS, &req)) {
if (EINVAL == errno) {
fprintf(stderr, "%s does not support "
"memory mappingn", dev_name);
exit(EXIT_FAILURE);
} else {
errno_exit("VIDIOC_REQBUFS");
}
}
/* check count */
if (req.count < 2) {
fprintf(stderr, "Insufficient buffer memory on %s\n",
dev_name);
exit(EXIT_FAILURE);
}
/* ALLOCATE */
/* allocate buffer pointers */
buffers = calloc(req.count, sizeof(*buffers));
if (!buffers) {
fprintf(stderr, "Out of memory\n");
exit(EXIT_FAILURE);
}
/* allocate buffers as requested */
for (n_buffers = 0; n_buffers < req.count; ++n_buffers) {
struct v4l2_buffer buf;
CLEAR(buf);
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
buf.index = n_buffers;
if (-1 == xioctl(fd, VIDIOC_QUERYBUF, &buf))
errno_exit("VIDIOC_QUERYBUF");
buffers[n_buffers].length = buf.length;
buffers[n_buffers].start =
mmap(NULL /* start anywhere */,
buf.length,
PROT_READ | PROT_WRITE /* required */,
MAP_SHARED /* recommended */,
fd, buf.m.offset);
if (MAP_FAILED == buffers[n_buffers].start)
errno_exit("mmap");
}
}
Uninitialization performs un-mapping the memory and releasing the allocated buffers.
static void uninit_device(void)
{
unsigned int i;
for (i = 0; i < n_buffers; ++i)
if (-1 == munmap(buffers[i].start, buffers[i].length))
errno_exit("munmap");
free(buffers);
}
Starting and Stopping of Streaming
After initialization, we will continue with start capturing function. Starting essentially does queuing the buffers with VIDIOC_QBUF and calls IO control 'VIDIOC_STREAMON'. That of stopping is 'VIDIOC_STREAMOFF' which are shown below.
static void start_capturing(void)
{
unsigned int i;
enum v4l2_buf_type type;
for (i = 0; i < n_buffers; ++i) {
struct v4l2_buffer buf;
CLEAR(buf);
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
buf.index = i;
if (-1 == xioctl(fd, VIDIOC_QBUF, &buf))
errno_exit("VIDIOC_QBUF");
}
type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
if (-1 == xioctl(fd, VIDIOC_STREAMON, &type))
errno_exit("VIDIOC_STREAMON");
}
static void stop_capturing(void)
{
enum v4l2_buf_type type;
type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
if (-1 == xioctl(fd, VIDIOC_STREAMOFF, &type))
errno_exit("VIDIOC_STREAMOFF");
}
Main Loop
In main loop function reads the number of frames set in global variable frame_count. The select() function is used to block the program until the frame is ready for the specified file descriptor or until a timer expires, whichever comes first. This facility is declared in the header file sys/types.h.
static void mainloop(void)
{
unsigned int count;
count = frame_count;
while (count-- > 0) {
for (;;) {
fd_set fds;
struct timeval tv;
int r;
FD_ZERO(&fds);
FD_SET(fd, &fds);
/* Timeout. */
tv.tv_sec = 2;
tv.tv_usec = 0;
r = select(fd + 1, &fds, NULL, NULL, &tv);
if (-1 == r) {
if (EINTR == errno)
continue;
errno_exit("select");
}
if (0 == r) {
fprintf(stderr, "select timeout\n");
exit(EXIT_FAILURE);
}
if (read_frame())
break;
/* EAGAIN - continue select loop. */
}
}
}
Reading frame
static int read_frame(void)
{
struct v4l2_buffer buf;
CLEAR(buf);
buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
buf.memory = V4L2_MEMORY_MMAP;
if (-1 == xioctl(fd, VIDIOC_DQBUF, &buf)) {
switch (errno) {
case EAGAIN:
return 0;
case EIO:
/* Could ignore EIO, see spec. */
/* fall through */
default:
errno_exit("VIDIOC_DQBUF");
}
}
assert(buf.index < n_buffers);
process_image(buffers[buf.index].start, buf.bytesused);
if (-1 == xioctl(fd, VIDIOC_QBUF, &buf))
errno_exit("VIDIOC_QBUF");
return 1;
}
Image Processing
After reading, the captured image is process in "process_image" function.
static void process_image(const void *p, int size)
{
fwrite(p, size, 1, stdout);
fflush(stderr);
fprintf(stderr, ".");
fflush(stdout);
}
We can modify the function to save as a pgm image instead of writing raw data as follow.
static void process_image(const void *p, int size)
{
char filename[] = "pic.pgm";
FILE *fp = fopen(filename, "wb");
size = size/2; /* pixel count is half of YUYV size */
int i;
unsigned char *cp = (unsigned char*)p;
for(i=0;i < size;i++){
cp[i] = cp[i*2];
}
if( fp )
{
fprintf(fp, "P5\n#\n640 480\n255\n");
fwrite(p, 1, size, fp);
fclose(fp);
}
}
In YUYV format (also called YUY2 in Windows environment), each 4 bytes is two Y's, a Cb and a Cr. [7]. Byte order for example 4 x 4 image is as follows.
start + 0: Y'00 Cb00 Y'01 Cr00 Y'02 Cb01 Y'03 Cr01 start + 8: Y'10 Cb10 Y'11 Cr10 Y'12 Cb11 Y'13 Cr11 start + 16: Y'20 Cb20 Y'21 Cr20 Y'22 Cb21 Y'23 Cr21 start + 24: Y'30 Cb30 Y'31 Cr30 Y'32 Cb31 Y'33 Cr31
To get greyscale image from that format, we can just extract even indexed bytes.
for(i=0;i < size;i++){
cp[i] = cp[i*2];
}
And write the resulting byte array with pgm image header to save as an portable graymap format (PGM) image as follows.
fprintf(fp, "P5\n#\n640 480\n255\n");
fwrite(p, 1, size, fp);
fclose(fp);
After executing the program, you can see pic.pgm image in the program folder.
Capturing BMP Image
The next step is to capture and save the image as bitmap image file. For that, create a new file 'ce_capture_bmp.c'. Firstly, data structures for image, bitmap header are declared. Then, functions to save, allocate, and free image are defined. And finally, I modify the process_image function, to save YUYV values as greyscale bitmap.
static void process_image(const void *p, int size)
{
size = size/2; /* pixel count is half of YUVU size */
int i,j;
unsigned char *cp = (unsigned char*)p;
unsigned char *po = (unsigned char*)malloc(size * sizeof(unsigned char));
process_raw(po,cp,size);
for(i=0;i < im.Height;i++)
for(j=0;j < im.Width;j++){
im.Image[i][j]= po[i*im.Width+j];
}
SaveBMPFile(filename,im);
free(po);
}
The resulting program can be found at the following link.
https://github.com/yan9a/rpi/blob/master/opencv/ce_capture_bmp.c
References
[1] The kernel development community. Part I - Video for Linux API. 2016.url: https://www.kernel.org/doc/html/v4.9/media/uapi/v4l/v4l2.html.
[2] LinuxTV. V4l-utils. 2017 Sep 14.
url: https://www.linuxtv.org/wiki/index.php/V4l-utils.
[3] Raspberrypi.org. Using a standard USB webcam. 2021.
url: https://www.raspberrypi.org/documentation/usage/webcams/.
[4] Gstreamer. Installing on Linux. 2021.
url: https://gstreamer.freedesktop.org/documentation/installing/on-linux.html?gi-language=c.
[5] Gstreamer. Installing on Linux. 2021.
url: http://manpages.ubuntu.com/manpages/cosmic/man1/gst-play-1.0.1.html.
[6] Derek Molly. Exploring Raspberry Pi. 2016.
url: http://exploringrpi.com.
[7] linuxtv.org. V4L2_PIX_FMT_YUYV ('YUYV'). 2021.
url: https://www.linuxtv.org/downloads/v4l-dvb-apis-old/V4L2-PIX-FMT-YUYV.html.
[8] linuxtv.org. V4L2_PIX_FMT_YUYV ('YUYV'). 2021.
url: https://www.linuxtv.org/downloads/v4l-dvb-apis-old/V4L2-PIX-FMT-YUYV.html.
No comments:
Post a Comment
Comments are moderated and don't be surprised if your comment does not appear promptly.