Introduction
In the last years, Haar LBP and HOG cascades for OpenCV became popular thanks to the even more satisfying detection capabilities of a wide range of objects. The success is motivated by the efficiency and the capability of this approach to detect rigid objects in real time under different conditions and on low-cost ARM architectures (such as smartphones, RaspberryPi, Arduino, etc.) as well. In the last years we collects dozens of Haar LBP and HOG cascades for OpenCV (versions: 2.x, 3.x, 4.x). To build a real Haar cascade it is necessary to burn the PC approximately for a week (often) with more than 32 Gb of RAM allocated. Obviously you can train a HAAR cascade with 500 samples P/N, 20 stages and a sub-space of the features. This is the right way to get in touch with the problem, not to develop something of real for the marketplace. Developing a real world cascade is still challenging (March 2020) but the results can be surprising even compared to deep neural networks techniques.
Want to download for free one of our cascades? Take a look below!
Why re-training the Haar LBP HOG cascades for OpenCV?
There are a lot of reason to re-train a cascade, the most relevant are:
- Your object is not covered by OpenCV repository (i.e. your specific targets).
- Latest version of OpenCV stopped the development, support and improvement of haartraining and Haar LBP and HOG traincascade applications. We are still working on it to support our customers worldwide.
- You need to handle a wider variability or different situations (i.e. face poses, different light conditions, etc.). Our datasets are collected (with a semi-supervised approach) from Tumblr, YouTube, etc. discovering the data variability more than usual. The OpenCV Haar LBP and HOG cascades come from the (not so near) past, 5/10 years ago the datasets were poor in variability due to the difficulty in accessing to the “fresh” samples.
- Shortage in CPU/RAM usually lead to a “sparse” features allocation. Our cascades are trained with a x64 machine with 16 cores and 64Gb RAM (but we are able to use a monster machine with 64 cores and 256Gb RAM, in this case it’s something near to the “full” features-space allocation, not just a subspace).
- With a few of modifications of OpenCV, the “traincascade” routine can be exploited to reach better performance. The biggest issue about speed is linked with the cascade structure, features selection strategy, dataset’s taxonomy, etc.
- Need an ARM detection suite with no floating points? Contact us!
- HAAR features are outperformed by other features both in efficiency and accuracy for the most of the “commercial” needing. Then it’s suggested to re-train an own cascade to exploit at the best the “new” features capabilities (LBP / HOG).
- You need a faster and reliable detector, to be better than your competitors (companies usually “wrap” OpenCV detector within their commercial products).
Performance
The Haar LBP and HOG cascades can be excellent both in efficiency and accuracy. With a few of modification in the training steps our cascades run up to +250% faster and the reliability can grow up to a 10x factor (both in false positives and false negatives). Our cascades have been trained with a modified version of OpenCV that include a lot of optimization, improving the accuracy and the speed of the detection. Generally speaking the speed of an object detector is heavy dependent from a lot of parameters (ie. min face detected, scale progression), architecture (i.e. ARM, x64), field of application (i.e. first person POV, video-surveillance, retail experience), etc. If your goal is detecting and tracking an object, the detection step is usually slower than the tracking step. In multi-object detection a “re-initialization” step is anyway necessary, so the full-image scanning is periodically mandatory (when just one-face at time is supported, a lot of optimization can be done to hyper-speed the detection and tracking). Regardless of these premises, building a fast and reliable cascade is the first – and fundamental – step. No real-time systems can be deployed on ARM or mobile device with a slow cascade, the OpenCV “free” cascades are extremely inefficient and not accurate.
Our boosted cascades run easily @30 FPS under Raspberry PI (mono-core, 700 MHz) with 2 USB cameras.
Despite what you may think the HAAR LBP HOG cascades are extremely more robust to the adversarial attacks, take a look to the clip below. In green our LBP cascade (30 fps with a raspberry PI 2) versus a deep neural network (in blue, ~1.5 fps on the same device). Our detector works perfectly, the DNN works poorly due to the adversarial attack.
It follows our benchmarks on ~30,000 images with faces in real scenarios (frontal, semi-frontal, occlusions, etc.). Our cascade outperforms drastically the OpenCV ones in every tested configuration. Take a look to the end of the page for the full results.
model: 'ARGO Vision frontal face cascade.xml' scale: 1.1000 neighs: 2 accuracy: 0.9950 sensitivity: 0.9950 fmeasure: 0.9975 MISSED: 180 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.1000 neighs: 2 accuracy: 0.3795 sensitivity: 0.3795 fmeasure: 0.5502 MISSED: 23194 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.1000 neighs: 2 accuracy: 0.4929 sensitivity: 0.4929 fmeasure: 0.6603 MISSED: 18956 Full results available at the end of this page.
Usage
Our Haar, LBP, HOG cascades are compatible with every OpenCV versions starting from 2.4.4. Updated list:
License and Commercial Use
At the moment more than a dozen of haar, lbp, hog cascades are involved in 9 commercial products. The “full” boosted cascades are available for commercial use, we are also available to train every brand new boosted cascades you need for your specific projects.
*NEWS*: since June 2016 vision-ary project joined ARGO Vision, an innovative firm that excels in visual recognition. For info about the cascades, please send a message via contact form here.
For any other info about Computer Vision and Artificial Intelligence please contact ARGO Vision.
Pre-trained Cascades (ready to download)
- Cats: LBP | HAAR | HOG
- Pedestrians: LBP | HAAR | HOG
- Frontal Face: LBP | HAAR | HOG
- Frontal Car: LBP | HAAR | HOG
- Eyes: LBP | HAAR | HOG
Trained Cascades (for specific projects)
- Rear Car: LBP | HAAR | HOG
- Profile Car: LBP | HAAR | HOG
- Bikes: LBP | HAAR | HOG
- Road Signs: LBP | HAAR | HOG
- Profile Face: LBP | HAAR | HOG
- Generic Face: LBP | HAAR | HOG
- Mouth: LBP | HAAR | HOG
- Nose: LBP | HAAR | HOG
- Hand: LBP | HAAR | HOG
- Nails: LBP | HAAR | HOG
- Shoes: LBP | HAAR | HOG
OpenCV reference: OpenCV official
Full results about Face detection problem
——————-
model: 'ARGO Vision frontal face cascade.xml' scale: 1.2000 neighs: 2 accuracy: 0.9940 sensitivity: 0.9940 fmeasure: 0.9970 MISSED: 230 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.2000 neighs: 2 accuracy: 0.3182 sensitivity: 0.3182 fmeasure: 0.4828 MISSED: 25487 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.2000 neighs: 2 accuracy: 0.4336 sensitivity: 0.4336 fmeasure: 0.6049 MISSED: 21174 ------------------- model: 'ARGO Vision frontal face cascade.xml' scale: 1.3000 neighs: 2 accuracy: 0.9916 sensitivity: 0.9916 fmeasure: 0.9958 MISSED: 320 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.3000 neighs: 2 accuracy: 0.2406 sensitivity: 0.2406 fmeasure: 0.3878 MISSED: 28389 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.3000 neighs: 2 accuracy: 0.3484 sensitivity: 0.3484 fmeasure: 0.5168 MISSED: 24358 ------------------- model: 'ARGO Vision frontal face cascade.xml' scale: 1.1000 neighs: 3 accuracy: 0.9947 sensitivity: 0.9947 fmeasure: 0.9973 MISSED: 201 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.1000 neighs: 3 accuracy: 0.3552 sensitivity: 0.3552 fmeasure: 0.5242 MISSED: 24104 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.1000 neighs: 3 accuracy: 0.4720 sensitivity: 0.4720 fmeasure: 0.6413 MISSED: 19737 ------------------- model: 'ARGO Vision frontal face cascade.xml' scale: 1.2000 neighs: 3 accuracy: 0.9932 sensitivity: 0.9932 fmeasure: 0.9966 MISSED: 258 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.2000 neighs: 3 accuracy: 0.2931 sensitivity: 0.2931 fmeasure: 0.4533 MISSED: 26425 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.2000 neighs: 3 accuracy: 0.4104 sensitivity: 0.4104 fmeasure: 0.5819 MISSED: 22041 ------------------- model: 'ARGO Vision frontal face cascade.xml' scale: 1.3000 neighs: 3 accuracy: 0.9885 sensitivity: 0.9885 fmeasure: 0.9942 MISSED: 433 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.3000 neighs: 3 accuracy: 0.2096 sensitivity: 0.2096 fmeasure: 0.3465 MISSED: 29548 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.3000 neighs: 3 accuracy: 0.3256 sensitivity: 0.3256 fmeasure: 0.4912 MISSED: 25211 ------------------- model: 'ARGO Vision frontal face cascade.xml' scale: 1.1000 neighs: 4 accuracy: 0.9941 sensitivity: 0.9941 fmeasure: 0.9970 MISSED: 221 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.1000 neighs: 4 accuracy: 0.3368 sensitivity: 0.3368 fmeasure: 0.5039 MISSED: 24792 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.1000 neighs: 4 accuracy: 0.4563 sensitivity: 0.4563 fmeasure: 0.6267 MISSED: 20324 ------------------- model: 'ARGO Vision frontal face cascade.xml' scale: 1.2000 neighs: 4 accuracy: 0.9915 sensitivity: 0.9915 fmeasure: 0.9957 MISSED: 317 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.2000 neighs: 4 accuracy: 0.2719 sensitivity: 0.2719 fmeasure: 0.4276 MISSED: 27217 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.2000 neighs: 4 accuracy: 0.3925 sensitivity: 0.3925 fmeasure: 0.5637 MISSED: 22709 ------------------- model: 'ARGO Vision frontal face cascade.xml' scale: 1.3000 neighs: 4 accuracy: 0.9846 sensitivity: 0.9846 fmeasure: 0.9922 MISSED: 576 ------------------- model: 'lbpcascade_frontalface.xml' scale: 1.3000 neighs: 4 accuracy: 0.1861 sensitivity: 0.1861 fmeasure: 0.3138 MISSED: 30425 ------------------- model: 'haarcascade_frontalface_default.xml' scale: 1.3000 neighs: 4 accuracy: 0.3053 sensitivity: 0.3053 fmeasure: 0.4677 MISSED: 25971
only for test
testing is the first step to make it works 🙂
Hi
Do you have somekind of tutorial how are you training these cascades? I would like to train my own with opencv but my current hit rate is very poor. I have 1600 positive images and about 4000 negative images of license plates and cars. I got about 86% hit rate on training set and lower than 50% on test set.
Best regards,
Kälver
I have no tutorial at the moment, you are experiencing the overfitting of your positive dataset (in training) or a bad generalization of the dataset because the training set doesn’t cover the “semantics” of the cars. Moreover 1600 samples are not so many, I trained my (commercial) last car cascade with ~100.000 positives randomly sampled.
I’m available for consultancy, contact me at info {{AT}} “nameofthesite”.net
Regards,
vision-ary
Hi, all this looks great. Thanks for posting your work and teaching articles.
I tried to use the car truck model with opencv 2.4.10 and got no results. Even using the picture on your site: http://www.vision-ary.net/wp-content/uploads/2015/06/cars_and_trucks_vision-ary.net_.png
I’m using the model like this:
CascadeClassifier detector;
detector.load(“../../Models/visionary.net_cars_and_truck_cascade_web_HAAR.xml”);
detector.detectMultiScale( image, detections,1.1,1, 0|CV_HAAR_SCALE_IMAGE,Size(40, 40));
Any thoughts on this??
Thank you very much.
Hello Matias,
thanks for sharing your experience.
I’m glad to help you, but I need to understand what do you mean with “no results”:
– no bounding boxes at all?
– no cars detected (the cascade works poorly)?
– software issues (check the detection time and report here) ?
Regards,
Alessandro.
Hi Alessandro, thanks for your reply. I insist, you are doing a great job here 🙂
Yeah that’s it, no bounding boxes detected, meaning no cars detected as well.
The model is loaded correctly, and it takes about 1 or 2 ms to analyse each picture (sometimes less, depending on the size of course). But no bounding boxes are detected.
Regards,
Alessandro.
Ok,
it looks something I already have experienced. There’s a “logical” bug in OpenCV that makes a cascade not suitable between different versions. I wasted a week some months ago to handle this bug, the “1-2 ms detection time” was exactly what highlighted the problem to me.
I’ll deep dive again to check it!
Gimme some days to check it. Regards,
A.
Hello Matias,
I double-checked the haar car cascade: http://www.vision-ary.net/2015/06/boost-the-world-car-detection/
As I highlighted to you the training process has little differences between opencv versions. Even if compatible, cascades won’t work between some different versions of opencv: as I wrote in the car cascade page the cascade is suitable ONLY for opencv < 2.4.3 (please open the cascade, is written inside as well). The reason why is tricky and it lead to AdaBoost C++ implementation within traincascade project. Please use opencv 2.4.2 to test the car cascade, I'm sure it works well (a similar one is within a commercial product). Unfortunately this cascade is the oldest one from the repository I've shared, it was trained with opencv 2.4.2. All the other cascades are trained with > 2.4.9 version.
In the future I will convert the cascade to latest “format” to join the “new” format.
Hello Vision-Ary,
we evaluated your cascades, they are superior in performance (both speed and accuracy). My company would like to evaluate the chance to include them in our computer vision commercial projects, how can we proceed?
Regards,
Michael.
Hello Michael,
thanks for contacting us!
We are evaluating offers from several companies, we will contact you soon by email to start a discussion about it.
Thanks,
Vision-ary team
Hello,
we would like to improve the detection approach to a wider class of objects keeping the best efficiency. Do you have any idea or suggestion?
Bye,
Sheila!
Hello Sheila,
thanks for contacting us!
We developed a multi-class version of the cascade approach for object detection and classification. Please contact us by the contact form for further information.
Regards,
Vision-ary team.
Hey!
We evaluated your cascades, amazing! My company would discuss with you the chance to make them suitable for our products on marketplace. How can we proceed?
Is the training process suitable for Linux env.?
Hello,
thanks for contacting us!
Please send an email by the contact form, we will share with you the info you need to start with the integration.
The training process is mixed Matlab/C++, suitable for Win and Linux.
Regards,
Vision-ary team
This article is really a good one it helps new internet visitors, who are
wishing for blogging.
Thanks!
This is our wish, help people getting close to computer vision 🙂
Regards,
Vision-ary team
Hi I am interesting in your cascade for my final project. Do you have a new cascade training?, my project need tracking and people detection in opencv but I only program and detect with Detect.Multiscale. Do you have another method for detect people with the webcam.
Thanks for your great job
Hello Carlos,
we have a lot of cascades, the most of them are in commercial projects now.
Please specify what do you need, detecting people is a common task in computer vision and there are a lot of methods.
Regards,
vision-ary team.
Hello,
Great work you have here, congratulations. I’m an engineering student from Brazil and I wonder if is it possible to use your face and eye lbp cascades on my final paper. I sent and e-mail for you with all the details and I appreciate if you take a look.
Thank you and continue this great job.
Best regards.
Do you mind if I quote a couple of your articles
as long as I provide credit and sources
back to your website? My blog site is in the very same niche
as yours and my visitors would truly benefit from a lot of the information you provide
here. Please let me know if this okay with you. Appreciate it!
Hello,
I am interested in using your commercial cascade (HOG or Haar) for car detection. Could you please get back to me with more details?
Hello Parin,
you received details via email. Keep in touch soon!
Regards,
vision-ary team.
Hi,
Need to detect cars in all angles. Is this being worked upon. If yes, can you please let me know accordingly.
Regards.
Hello Amit,
we are able to do it! Send an email to us and we will reply to you about your project!
Vision-ary team
I am curios how does the boost cascades compare to using a CNN. With opencv 3.3 you can import an already built cafee, pytorch, etc model on a Raspberry pi and should get better results, I guess
I’m pretty sure you are right. CNN are extremely powerful if compared against cascades. The question is: is it meaningful detecting one class rigid object with a CNN (slow training, gpu dedicated, cpu load very relevant) if you can detect a face with a cascade with 2-300 MHz processor in real time? Cascade is still the best tradeoff speed/accuracy in our opinion.
The result comparison that you show it cannot be correct. Are the results that you are reporting tested on the train set? If it is not the case do you have any scientific publication about your detector? If you are getting something 150% faster and 10% more reliable it should be worth for a CV conference.
The results we reported are performed on test dataset completely separated from training set (proprietary dataset, not public).
Our goal is commercial, not research. This is why we have more a dozen commercial projects with clients disappointed about opencv performances.
Stay tuned for our research on deep learning, probably in 2018 we will publish something.
Hey! do you have pre-trained cascade for fire detection?
Hello! Not yet, but if you have a good dataset we can train for you in a couple of days. Send an email to us!
Regards,
Vision-ary Team