Mobile Tagging
How Good is the Latest Zabble Zero AI™ in Identifying Contaminants?
August 8, 2022
August 8, 2022
Friday, October 6, 2023
Zabble Zero Mobile Tagging™ was designed to help customers get more data points, faster, with AI-assisted fullness and contaminant tracking. Our algorithms predict the fullness of a receptacle from a photo, and until recently, suggested contaminants based on tagging history. Now, our latest app version includes a much-requested new feature: Zabble Zero AITM has object detection to identify items from a photo and suggest relevant ones as contaminants. How well does the new AI work?
To evaluate the new model and algorithm prior to release, we had 5 testers, on multiple occasions, meticulously run tests to validate the outcome of the AI across 10 categories of items the AI has been trained to detect.
The following process was followed:
An accurate result (true positive) for that category had to meet each of the following criteria:
Here is an example of a true positive result when testing cans in the landfill stream:
We adjusted our algorithm based on initial testing, and then conducted a final round of testing to report the following results.
As shown in the table below, categories that were most accurately detected have higher true positives (TP) and fewer false negatives (FN). Aluminum cans had a perfect score! Food, plates, and bottles were successfully detected only about half the time in our test.
Some categories had higher false positives, which means they were wrongly detected in the photo. For example, we saw 9 cases where paper/cardboard was detected but wasn’t actually in the photo. Overall, specificity was about 97.6%, which means our object detection model is good at knowing when items are NOT present in the photo.
The visual below shows which categories got mistaken for other items. For example, several different items (bottles, containers, cups, foil/plastic film, and food) were predicted as paper/cardboard at least once. On the other hand, even though our model wasn’t as good as detecting bottles or food, when it does detect them we can be pretty confident the prediction is correct - because no other item was wrongly labeled as a bottle or food in our test.
Note: Confusion matrix has been normalized so that rows sum to 1.
Objection detection in Zabble Zero AI™ will get smarter over time as we retrain our model with more images and different types of items. We’ll also add more categories and aim for higher accuracy at different distances.
Stay tuned for an update about how our latest AI performs in the field. And if you’re not already using Zabble Zero™, contact us for a demo.
Friday, October 6, 2023
1966 Tice Valley Blvd, #105,
Walnut Creek, CA 94595
Tel.: 925-289-9345
Email: team@zabbleinc.com