Build your own intelligent garden
Introduction
Gardening has always been a blend of science and art. But with the increasing challenges of urbanisation, climate change, and biodiversity loss, new innovative solutions are required. Enter the intelligent garden—a fusion of AI, IoT, and data-driven insights that revolutionises how we nurture our green spaces.
By integrating advanced sensors, cloud computing, coupled with AI-powered sensing and analytics, an intelligent garden becomes more than just a collection of plants. It transforms into a responsive ecosystem where every tree, flower, and insect play a part in a data-driven conversation about sustainability and growth.
Technology as a Solution
AI and sensor-based monitoring offer a revolutionary approach to urban gardening by providing real-time insights into urban greenspace health and environmental conditions.
The Avanade Intelligent Garden addresses these challenges by deploying an integrated system of sensors and AI-powered analytics to enhance urban tree survival rates and optimize maintenance strategies. By leveraging Cloud Services from Microsoft Azure and hardware from partners such as ePlant, it allows users to query garden data through natural language, democratising access to critical insights.
This data-driven approach enables custodians to make informed decisions, ensuring trees and plants receive the precise care they need to thrive. AI-powered solutions such as the Avanade developed "TreeTalk", allow trees to communicate their needs, flagging issues such as dehydration or environmental stress before they become critical.
By understanding the problem and leveraging intelligent technology, we can transform urban tree management from reactive care to proactive stewardship, creating healthier, more resilient green spaces for future generations.
The Avanade Intelligent Garden: A Case Study
The Avanade Intelligent Garden is an innovative approach to urban tree management, designed to showcase how artificial intelligence, environmental sensors, and cloud technologies can support the health and survival of trees in challenging environments.
Created in collaboration with Avanade and Microsoft, and designed by Tom Massey and Je Ahn, this project aims to enhance tree resilience, improve biodiversity, and promote more sustainable horticultural practices.
At the heart of the Avanade Intelligent Garden is a cutting-edge ecosystem of interconnected sensors and AI-driven analytics that continuously monitor tree and plant health as well as the presence of pollinating insects. Along with other important factors, the intelligent system addresses the critical issue of urban tree loss, where many trees fail to thrive due to extreme weather conditions, poor soil health, and inadequate water management. By providing actionable insights, custodians of urban trees can make informed decisions, improving the survival rates of city greenery.
Technological Integration
The intelligent garden utilises:
- Tree Sensors tracking growth, lean, humidty, and temperature.
- Soil Sensors to monitor soil moisture and acidity.
- Pollinator Camera Traps conducting Flying Insect Transit Counting to analyse pollination activity and support biodiversity.
- Temperature, Humidity, and Air Quality Sensors providing insight into microclimates affecting plant health.
- Water Sensors offering insights into the the condition of a body of water in the garden.
- A Web Front-End built using Svelte and utilising Microsoft Azure Services such as OpenAI, IoT Hub, CosmosDB, and Storage, enabling users to query and interpret real-time garden data using natural language.
These technologies work together to create a dynamic, responsive, and intelligent horticultural system, where trees can effectively "communicate" their health status, allowing for proactive maintenance and better resource efficiency.
AI-Driven Tree Care
A standout feature of the Avanade Intelligent Garden is TreeTalk, an AI-powered interface enabling visitors to interact with trees, learning about their current health, environmental conditions, and care needs. This human-centered AI approach ensures that urban planners, city councils, and green space custodians can make data-driven decisions that promote the long-term survival and well-being of urban trees.
Through projects like the Avanade Intelligent Garden, AI and IoT demonstrate their potential to revolutionise horticulture, improving not just the survival rates of trees and plants, but also promoting sustainability and enhancing community engagement in urban environments.
Components of an Intelligent Garden
Building an intelligent garden requires a combination of readily available hardware and trusted cloud-based services to enable seamless data collection, analysis, and interaction with garden insights. These components create a dynamic and responsive system, allowing trees, plants, and pollinators to be monitored and nurtured with precision.
For the Avanade Intelligent Garden, the team utilised a wide variety of sensors, microcontrollers, embedded PCs, communication technologies and Cloud Services.
This forms a well thought through and proven blueprint for others to base their designs on.
Hardware Infrastructure:
The intelligent garden features a diverse set of sensor-driven systems designed to gather crucial environmental data:
GEM System
- ESP32-based platform, programmed using nanoFramework for lightweight deployment.
- Captures temperature, humidity, air quality, and water sensing, ensuring optimal environmental monitoring.
- Transmits telemetry to Azure IoT Hub, enabling real-time visibility into changing conditions.
ePlant Tree Sensing System
- An ePlant technology TreeTag, leveraging LoRaWAN connectivity to transmit data to a gateway device.
- Sensor readings are stored in a third-party cloud service, which can be queried via API for comprehensive tree health data.
- An Azure Function continuously pulls readings from the API and forwards them to Azure IoT Hub for seamless integration with the broader system.
Raspberry Pi AI Pollination Monitoring ("Fit Counting")
- Built using Raspberry Pi 5 and Hailo 8 AI Kit+, written in Python for high-performance object detection.
- Captures stills and video, identifying flowers and insects, boxing detected objects for analysis.
- Uses AI-driven intersection detection to determine if pollinators have landed on flowers.
- Live telemetry is sent to Azure IoT Hub, while stills and video footage are processed via Azure Functions for front-end display.
- Web Front-End for Live Data Streaming displays real-time video streams, graphical insights, and data tables detailing garden telemetry.
- Integrates seamlessly with Azure services, offering interactive visualizations of sensor data.
Networking Hardware
- Leveraging a Router or Access Point capable of being connected to a USB 4G / 5G modem to allow for the important connectivity of devices to the cloud.
Cloud-Based Web Services
A robust suite of Azure-powered web services forms the backbone of the intelligent garden's data processing and interaction framework:
Azure IoT Hub
- Centralised service that facilitates sensor telemetry collection, ensuring smooth data ingestion and processing.
CosmosDB
- Serves as the primary data store for all sensor readings, enabling fast retrieval and integration with other Azure services.
Azure Container Apps
- The main user interface, built with Svelte and TypeScript, where telemetry data, insights, and interactive features are displayed.
- Hosts the TreeTalk application, allowing users to engage in AI-driven conversations with trees.
Azure OpenAI
- Making use of the OpenAI GPT4o mini model, which is a smaller model with a far smaller training footprint from a sustainability perspective.
- Translates raw sensor data into human-readable insights, generating meaningful summaries and responses based on live environmental conditions.
Azure Functions
- Handles various tasks, including:
- Data processing and analyses.
- Tree Tag API and other 3rd party data retrieval for real-time tree health and environment monitoring.
- CosmosDB Data Access Layer (DAL) for structured data management.
- Processing of Pollinator Camera stills and videos along with processing of telemetry data to return refined metrics and statistics around pollinator activity and surfacing using a Raspberry Pi web front-end portal and the primary displays.
- Developed using Node.js for optimized serverless execution.
- Handles various tasks, including:
Azure Storage
- Blob Storage for video footage and high-resolution images captured by pollinator cameras and tree-monitoring systems.
- Table Storage for metrics and statistics storage.
Weather API
- Querying of public API MET Office API data for local weather monitoring.
Together, these components enable deep environmental insights, streamline garden management, and enhance data accessibility via natural language queries. This fusion of AI and IoT-driven analytics empowers custodians to make data-backed decisions, improving plant survival rates and fostering biodiversity in urban spaces.
Shopping List of Parts and Cloud Services
Below is the hardware shopping list for building your Intelligent Garden;
Raspberry Pi Pollinator Counter
| Item | URL | Notes |
|---|---|---|
| Raspberry Pi 5 (8GB or 16GB) | thepihut.com | Primary processing unit for pollinator tracking. |
| 32GB minimum Class A1 SD Card | thepihut.com | Required for operating system and data storage. |
| Raspberry Pi Hailo 8 26 TOPS AI Kit+ | thepihut.com | AI acceleration for insect and flower detection. |
| Official Raspberry Pi 5 40W PSU | thepihut.com | Power supply for stable operation. |
| Raspberry Pi Camera Module 3 | thepihut.com | Captures pollinators and flowers in real time. |
| Raspberry Pi Camera Ribbon Cable | thepihut.com | |
| 3D Printed Camera Case | thingiverse.com | Houses the Raspberry Pi and Camera |
GEM System
| Item | URL | Notes |
|---|---|---|
| ESP32 C3 microcontroller | amazon.co.uk | Manages environmental sensors and data transmission. |
| AHT 21 Temperature, Humidity, Pressure and Air Quality Sensor | amazon.co.uk | Provides critical environmental monitoring. |
ePlant Tree Monitoring System
| Item | URL | Notes |
|---|---|---|
| ePlant TreeTag tree monitoring sensors | eplant.com | Tracks growth, water uptake, and structural health. |
| LoRaWAN Gateway Device | eplant.com | Connects tree sensors to the cloud. |
Networking
| Item | URL | Notes |
|---|---|---|
| WiFi Access point / Router | Connectivity for the various IoT Devices |
Microsoft Azure Web Services & Cloud Infrastructure
| Item | URL | Notes |
|---|---|---|
| Azure IoT Hub | azure.microsoft.com | Central telemetry hub for all sensors. |
| CosmosDB Subscription | azure.microsoft.com | Stores sensor data for retrieval and analysis. |
| Azure Web Apps Hosting | azure.microsoft.com | Provides user interface for querying garden data. |
| Azure OpenAI Subscription | azure.microsoft.com | AI-driven natural language processing for data insights. |
| Azure Functions Deployment | azure.microsoft.com | Serverless functions for sensor data processing. |
| Azure Storage Subscription | azure.microsoft.com | Storage for pollinator videos and environmental logs. |
Methodology
The Avanade Intelligent Garden collects sensor data from various sources and stores it in Cosmos DB. This data includes environmental readings such as temperature, humidity, pressure, water quality, and air quality, which are gathered by ESP32 devices connected to sensors. These devices send the readings to an IoT Hub on a regular basis for display on the primary Web Front End.
Web Front End
The Web Front End, written in Svelte and TypeScript, surfaces all sensor data in a holistic manor, bringing together the various sources of data and presenting them to the user in a manor allowing for easy consumption and understanding.
The Web Front End queries the data via Azure Functions to surface the information which follows a micro-service architecture.
TreeTalk Application
The TreeTalk app works by taking the user's query and using it to query the database. The results are then combined with the original query, and OpenAI is used to provide a human-readable response to the original question using the retrieved data.
This AI-powered interface enables visitors to interact with trees, learning about their current health, environmental conditions, and care needs.
ePlant Tree Sensors
The ePlant Tree sensors are screwed with care into each tree. These sensors are solar powered, and sense data such as the lean angle of the tree, the moisture, temperature and so on.
This data is delivered to a gateway via LoRaWAN. The gateway is installed nearby and collects the readings from each sensor and delivers that data to the ePlant cloud services.
ePlant then make available an API endpoint, from which data can be collected. This endpoint is queried using an Azure Function configured on a regular timer. The collected data is stored in the CosmosDB database for querying by the rest of the system.
Pollinator Camera
For the Pollinator Camera, we train a YOLO model with captured images of flying insects and flowers 10. The model is run on a Raspberry Pi 5 using the AI Kit+ Hailo Neural Processing Unit (NPU) to detect the presence of flying insects and flowers. Bounding boxes are drawn around detections, and stills and videos are captured and stored in Azure Storage.
Overlapping bounding boxes between flowers and flying insects are inferred to indicate that the insects have landed on the flowers. Statistics for the number of insects, times, dates, and whether they have landed are collected, stored, and surfaced along with stills on a local front end hosted on the Raspberry Pi. Each detection is also sent to an IoT Hub along with the filenames of stills and videos for display on the primary Front End.
Soil, Temperature, Humidity, Pressure, Air Quality and Water Quality Sensors:
The local environment of the garden is monitored with a slew of sensors all connected to ESP32 Microcontroller devices.
These devices collect various points of data and deliver them on a regular occasion to an IoT Hub.
The IoT Hub in turn triggers an Azure Function which stores the collected data in the CosmosDB Database for surfacing by the primary Web FrontEnd.
These devices are low powered by AA batteries, and are in fact in sleep mode for the majority of the time, only waking for short periods to record data and deliver it to Azure.
Local Weather
The local weather data is fetched from the MET Office API and displayed live on the primary FrontEnd of the Intelligent Garden system. The process uses an Azure Function to query the public MET Office API for local weather data.