Non-destructive tomato sizing with stereo vision

What this project is

This project is part of my MS thesis at Gebze Technical University, within the TÜBİTAK 1071 / ARDEB 124N905 bilateral R&D project between Türkiye and Pakistan (GTU, NUST, Cx Technology and FFC-FACE). I own the machine learning workstream, and fruit sizing is one of its core computer vision tasks.

Two calibrated cameras photograph the plants every 30 minutes. The pipeline detects each fruit and its ripeness, measures its diameter from stereo depth, follows every fruit across months, and writes growth curves, a timelapse video and a CSV that feeds the project dashboard.

What needed solving

Picking a tomato just to measure it wastes the fruit, so the system measures it right on the plant. Fruit size, count and ripeness over time show how a crop is growing and when it will be ready to pick. Measuring by hand means someone entering the greenhouse and handling the fruit, which is slow, patchy and disturbs the plants.

This project replaces that with a fixed stereo rig that measures every visible fruit every 30 minutes, with no contact. The per-fruit size, count and ripeness feed the project dashboard, which supports harvest and growth decisions and the project target of at least 10% more yield than conventional greenhouses.

What I did

• Own the machine learning workstream within the 24-month bilateral project.
• Built the end-to-end pipeline, from stereo capture to per-fruit growth curves, a timelapse and a CSV for the dashboard.
• Trained and compared YOLOv10 ripeness detectors on tomatOD and a merged dataset, and tuned them with Ray Tune.
• Calibrated the stereo pair with ChArUco boards and worked around greenhouse glare and leaf occlusion.
• Designed the three-model comparison framework (rule-based, RGB-only CNN, RGB+Depth fusion) that will form the basis of a conference paper.
• Presented the work to my supervisor (April 2026), at the GTU graduate seminar (Spring 2026) and at a GTU bilateral workshop on greenhouse dashboards and digital twins.

I work under Prof. Dr. Yusuf Sinan Akgül, with plant scientists from the GTU Molecular Biology and Genetics department who host the greenhouse and grow the plants, and with the NUST team that builds the HarvesTrack dashboard.

How it works

1. Capture

   Two cameras at 5 MP (2592 × 1944) take a stereo pair every 30 minutes, with auto-exposure stabilisation. Pairs are matched by timestamp across about three months.

2. Stereo calibration

   A ChArUco board (6 × 9, 28 mm squares) calibrates the pair in OpenCV. Reprojection error is about 0.5 px per camera and 1.05 px for the pair.

3. Rectify and depth

   Images are rectified and a disparity map gives depth for every pixel, using classical block matching and SGBM, with RAFT-Stereo as the deep alternative.

4. Detect

   A YOLOv10 detector finds every fruit and classifies it as unripe, semi-ripe or fully ripe.

5. Measure

   Diameter = box width in pixels × median depth ÷ focal length. The box covers the whole fruit, so a leaf in front does not shrink it, and the median depth ignores outlier pixels.

6. Track

   Centroid matching links detections over time, then KMeans clustering merges fragments into one stable ID per fruit. Frames washed out by strong daylight (07:00 – 15:59) are skipped.

7. Output

   A CSV with timestamp, fruit ID, ripeness, diameter, depth and confidence, plus annotated frames, a 10 FPS timelapse and growth charts for each fruit.

8. Three-model comparison

   A framework comparing a rule-based method, an RGB-only CNN and an RGB+Depth fusion model on the same sizing task.

The sizing method took three attempts to get right:

White-balance corrections were also tried for the daytime glare. None beat simply skipping those frames.

Tools and methods

What came out of it

Fruit detection, trained on tomatOD alone and on tomatOD merged with a Kaggle ripeness dataset:

Merging more varied data helped more than tuning hyperparameters. Models trained on tomatOD alone lose most of their accuracy on the differently styled Kaggle images, so cross-domain robustness is the main open problem for detection.

The sizing comparison puts three approaches side by side on the same task:

• Simple rules: a rule-based method with no learning.
• RGB-only CNN: a network that learns fruit size from the colour image alone.
• Colour plus depth: a deep learning model that uses both RGB and depth.

Full results will appear in the conference paper on non-destructive tomato sizing (in preparation, 2026).

Still open:

• Run the full three-month dataset (about 3,000 stereo pairs). So far the pipeline has been run on 200 – 500 frame batches.
• Raise semi-ripe precision, currently the weakest class (46.8%).
• Domain adaptation for new camera setups, then deployment on a Jetson edge device.
• Combine size and ripeness with crop growth models for yield estimation.