Drones Footage Analysis with Computer Vision
In this case, we worked with a client who wanted to understand how AI and Computer Vision technologies could possibly support them in their tree-planting and forests monitoring projects. Thus we developed an engine to offload their teams with the majority of the manual work related to complex analysis of images. Initially the team was taking pictures of the forests with drones. Later on experts were analyzing the images to detect anomalies (i.e. drying out trees) and count trees (especially the young ones to assess the survival rate).
We started our collaboration with a set of workshops, explaining the client what’s possible wite latest technologies and how machine and deep learning could possibly help them address some of their challenges and ideas. We ended up with a proof of concept build-up to address the first scenarios we jointly nominated as first candidates for automation.
We built a software engine that made it possible to use images taken by drones to count young trees and detect anomalies with the accuracy of 90%+.
Drones flying over the area produced hundreds of 4K images. We used them to build a data set which became a foundation for neural networks training. We initially trained the network for 3000 epochs which helped us detect around 221 trees per picture (initial results). Expanding the training to 9000 epochs let us increase the number of found trees to 285 trees, translating into accuracy of 92-95% (depending on the area). We also trained the network to detect basic anomalies like dry trees.
Benefits for the customer
- Understand how technology can help them become more efficient and address many problems
- Ability to perform analytics using the images generated by drones and find problematic areas
- Significantly reduce the time of analyzing large areas
We started off by analyzing the data we were able to collect with a help of drones flying over the area. They produced hundreds of 4K images. So the natural first step for us was to review the pictures and process them to build a data set which would become a foundation for neural networks training. Due to the high resolution of the images, we decided to split the single 4K images into the set of smaller images. In fact, that allowed us to avoid resizing of the images, and hence to maintain an appropriate quality of the training data. Once we accomplished that, we started with the DarkNet framework to train our deep learning engine. The deep neural network which we used consisted of 27 layers, including 22 convolution layers an 5 max pooling layers. The next step was to train the network and optimize it for efficiency. Despite many things like ensuring the proper flow of data thru the network, the key challenge was to obtain a set of weights that would help us configure the network and get best possible accuracy. The network was receiving transformed images produced by drones, analyzing them and tagging the young trees. We initially trained the network for 3000 epochs which helped us detect around 221 trees per picture (initial results). Expanding the training to 9000 epochs let us increase the number of found trees to 285 trees, translating into accuracy of 92-95% (depending on the area). We tested our solution with very similar images we received from another client and the results were very much comparable.
The solution addresses the key pains of the IoT applications:
- Enable Scalability
(Decentralizes AI services & makes it easier to expand the IoT ecosystems)
- Enable near-real-time AI experience
(By using the modern low power, high performance, small form factor accelerators)
- Solve round-trip latencies
(Deploying AI directly on the device enables making on-the-spot decisions)
- Eliminate intermittent connectivity related issues
(No need for sending the data from the device to external AI services and waiting for results)
- Reduce costs of bandwidth
(AI-enabled devices pre-process the data and send the results to external services vs raw data)
- Data can stay locally on the device
(Having AI on the device allows for sending the data to external storages selectively)