VN September 2025

Vetnews | September 2025 32 « BACK TO CONTENTS Thus, based on our results, if resources on the ground are very constrained, it could be helpful to prioritise thermal sensors for midden detection. Efficient Querying with Active Learning The passive learning systems achieve high accuracy but require 9,736 labelled images, so such systems cannot be used to map middens in other sites where we do not have any of their coordinates beforehand. Moreover, differing topography among sites makes transfer learning challenging. Labelling enough images for passive learning systems to be accurate would take domain experts dozens of hours. In this subsection, we demonstrate that our MultimodAL algorithm, described in Subsection 4.1, is able to match the best passive learning performance with 94% fewer labelled images by employing an efficient query strategy. While we do have the ground-truth midden locations for this site, enabling us to test various detection methods, we use the results hereinafter as a proof of concept for our system. For active learning, we use the same dataset and perform the same train-test split procedure as for passive learning, except that we do not balance the training set, so all the empty images not set aside for testing (9,665) are available to the system during training. We test two versions of MultimodAL. First, MultimodAL: Thermal+RGB Fused uses the version of our proposed algorithm for a single model trained on the pre-fused thermal+RGB images, and second, MultimodAL: Thermal + RGB maintains separate models trained on the thermal and RGB images and uses the version of the algorithm that accommodates multiple modalities. In addition to our proposed MultimodAL algorithm, we implement several baselines: (1) Random: randomly chooses images to be labelled in each round, (2) Uncertainty: requests labels for the images with model outputs closest to 0.5, a very common active learning strategy [Lewis and Gale, 1994], (3) Positive Certainty: requests labels for the images with outputs closest to 1, an active learning strategy more tailored to imbalanced datasets like ours [Kellenberger et al., 2019], and (4) Disagree: asks the user to label the images with the greatest difference in output between the thermal and RGB models [Zhang et al., 2021]. We train and evaluate the single-model systems on the thermal+RGB fused data since that model performed the best in the passive setting. For the multi-model systems, we use separate thermal and RGB models. We train each of these on the images in its modality. In evaluating the system as a whole, images in the test set are classified by weighting the models’ outputs by their weights learned during training, calculated using Equation 2. We graph the accuracy of our proposed methods in comparison to the baselines as the labelling budget increases in Figure 7. We find that both versions of MultimodAL outperform all of the baselines by a statistically significant amount (p < 0.05) at 500 labels. This is despite the fact that Positive Certainty is statistically significantly better (p < 0.05) than Random, demonstrating it is a strong baseline. Positive Certainty outperforming the other baselines is consistent with our hypothesis that prioritising images the system thinks are positive samples is more suitable for highly imbalanced datasets than prioritising images about which the system is uncertain. Even so, it is the two variants of MultimodAL that get within 0.026% of the best passive learning accuracy by 500 labels, which is not a statistically significant difference (p > 0.05). Hence, there is a negligible difference in performance despite the 94% difference in the number of labelled images provided to the passive system and MultimodAL. Interestingly, MultimodAL: Thermal + RGB outperforms MultimodAL: Thermal+RGB Fused for small numbers of labels (< 200), despite the opposite pattern holding for larger numbers of labels. Because MultimodAL: Thermal + RGB combines the scores across two modalities, it will more strongly prioritise images that are very clearly middens in both imagery types than the single-model MultimodAL: Thermal+RGB Fused, which will assign a high score to images that are clear in either data type. This could result in more informative midden images being selected in the former case and leading to improved performance when the number of discovered middens is low. Overall, the advantages of MultimodAL are greatest when the labelling budget is very small (< 150). For example, we see that for up to 50 images, labelled only MultimodAL does better than Random. Furthermore, we see diminishing returns in accuracy for MultimodAL after around 200 labels once the system has been trained on enough middens (∼ 33) to serve as a reasonable detector. To dig deeper into the origin of MultimodAL’s performance gains over the baselines, we consider another important evaluation metric: the fraction of middens discovered during querying by the active learning system out of all those present in the training set as more labels are provided. Figure 8 shows that both versions of the MultimodAL system are able to find more middens more quickly than the baselines. At around 400 labels, the Positive Certainty model is accurate enough to compete with MultimodAL in terms of midden finding despite maintaining a lower test set accuracy, but for smaller numbers of labels, its midden retrieval is significantly inferior. T+R Fused Thermal T+L Fused T+R+L Fused RGB R+L Fused LiDAR Accuracy .864± .008 .862 ± .012 .858 ± .008 .818 ± .011 .632 ± .014 .608 ± .016 .595 ± .012 Precision .875± .010 .846 ± .014 .826 ± .011 .823 ± .012 .629 ± .015 .616 ± .020 .632 ± .023 Recall .855 ± .016 .894 ± .016 .917± .011 .818 ± .017 .670 ± .023 .678 ± .028 .576 ± .041 F1 .861 ± .009 .866 ± .012 .866± .007 .817 ± .012 .642 ± .015 .631 ± .015 .565 ± .024 Table 1: Passive learning statistics. T=Thermal, R=RGB, and L=LiDAR. The Thermal+RGB (T+R) Fused model achieves the best performance on accuracy and precision, and the Thermal+LiDAR (T+L) Fused model achieves the best recall and F1 score Figure 7: Mean accuracy on the test set for the active learning methods with up to 500 images labelled across 30 trials. Both versions of MultimodAL outperform the baselines and, with 500 labels, are statistically indistinguishable from the best passive method Figure 8: Mean fraction of middens found in the training set for the active learning methods with up to 500 images labelled across 30 trials. Both versions of MultimodAL find more middens more quickly than the baselines with up to 400 labels

RkJQdWJsaXNoZXIy OTc5MDU=