ViTech is a technology transfer experiment conducted within the frame of the SMART4ALL project between partners from Serbia and Montenegro in the field of digitized agriculture. It tackles AI-enabled cyber physical systems embedded in low altitude platforms, such as drones and vehicle-mounted cameras to perform efficient, low power, on-the-fly image processing algorithms in order to bring novel CPS functionalities and provide multi-modal perception for autonomous peri-urban agricultural robotics. Depth perception solution allows recognition and classification of various 3D object in outdoor environments and enable differentiation of agricultural vegetation of interest in the complex environments where metropolitan and rural sceneries are interwoven, and where classical AI-based approaches fail to deliver satisfactory results. The solution that has been developed can easily be scaled up for a number of different applications as it allows running an arbitrary AI algorithm for video and image processing on an off-the-shelf platform. The technology was validated in field experiments on several locations in Montenegro and Serbia.

Objectives

ViTech aimed to create a service that would empower operators of airborne and vehicle-born visual platforms to analyze 3D scenes in peri-urban areas to unlock their suppressed agricultural potential and aid peri-urban farmers with tools otherwise accessible only to large landowners. ViTexh covers the competence fields of AI inspired agriculture, UAV based agricultural and plant monitoring, Smart farming, Agricultural decision making systems and Selective plant protection

Challenges

Visual perception is the foundation of vision-based inference. Thus, it represents the key enabling technology for mobile robotics, outdoor navigation, autonomous driving and many other contemporary CPS that are finding their way in digitized agriculture. Embedded vision platforms are the next step in the design and development of devices that provide such capabilities. They offer seamless integration in existing platforms and are highly optimized for the execution of computer vision algorithms. Small dimensions and efficient use of resources, at low cost, are always contrasting performance and diversity of capabilities that device offers. In that sense, it is hard to find a platform that would be a one-size-fits-all solution. General vision tasks ’in the wild’ are always considered as hard and the AI semantic analysis of 3D scenes has been a challenging and unanswered issue for many years. ViTech technology is capable of providing uncertainty reduction by implicit or explicit use of 3D information and is expected to bring complex vision functionalities to agricultural sensor systems operating in uncontrolled environments. It will enable the product positioning among different solutions competing on the market. Embedded platforms capable of providing necessary level of 3D perception with low energy consumption will be a preferred choice for solving vision tasks in all future CPS. The project raised the level of the technology from TLR 4 to TLR 7.

Technology

The ViTech PAE has made significant breakthrough in design of generic and modular computer vision gateway, capable of adapting to different user need and application scenarios. It is a fully customizable tool for design of real-time adaptive processing pipelines, according to user needs and with minimal programming effort. Also, it is a software package tested on commercially available hardware, which was shown to be easy for integration with commercially available drone platforms. In particular, developed system and test results provide the technology receiver with following capabilities:Full control over configuration of each of the functional blocks reflecting original system design requirements Ability to easily extend different parts of the system, due to modular design and use of open source software Ability to easily integrate different in-house solutions or external, proprietary systems and sensor devices into real-time processing pipelines created by developed vision gatewayOvercome the need of using complex, proprietary stream processing libraries, in order to achieve the same level of functionalitySupport different types of input sources, as well as different types of input stream formats in unified and systematic way (e.g. by utilizing local input streams coming from camera mounted on drone, agricultural machinery, or vehicle; or remote signals streamed to ViTech gateway over the network) Capability to exploit hardware encoding and decoding on the ViTech gateway for any combination of user defined multiple input and multiple output streams Capability to perform resource efficient deployment of AI models in outdoor environments, by utilizing heterogeneous computing of ViTech gateway and provided set of model translators and compilers enabling creation of optimized AI inference engines. Stream processing and AI deployment with low power consumption and low memory footprint Versatile tool for performing both low-level and high-level vision tasks Real-time monocular depth perception on drone platform

Market

Peri-urban agriculture constitutes 15% of the overall agricultural sector in the EU and is on the constant rise. It is recognized for its importance by policy makers since 1991 by the UN Development Programme, and in 21st century it saw the rise of new generation of small urban and peri-urban farmers gaining awareness and interest in its advantages regarding food quality, transparency, traceability, security and local production, as well as financial benefits from various support programmes. Due to the UN-UNEP and UN-FAO Sustainable Food Systems Programme, through the New Urban Agenda by the UN, Global partnership on sustainable urban agriculture and food systems, and PURPLE– peri-urban regions platform Europe, all the way to the provisions of the EU’s Green Deal, the subsidies have increased 100 times in the EU in the past 25 years. This is mirrored in the research funding, going from 3.2 mil within the FP5, to 10 mil in FP7 and 76 mil in H2020, all the way to 120 mil in the Horizon Europe programme. In spite of the scale of the support, there are still very few end-to-end digitized solutions on the market. The competition in the domain of renting airborne vision systems is in the expansion but it offers no custom functionality beyond the commercially available tools. Some custom solutions without GPU using an OAK-D smart camera provide vision capabilities necessary for autonomous drone navigation, but no major brand of drone manufacturer offer their own solution for AI-driven vision systems for agriculture.

Consortium