Krishnan, G., Hofmann, H., “Adapting the Chumbley Score to Match Striae on Land Engraved Areas (LEAs) of Bullets.” Journal of Forensic Sciences, Vol. 64, No. 3, 2019, pp. 728-740.
The same‐source problem remains a major challenge in forensic toolmark and firearm examination. Here, we investigate the applicability of the Chumbley method (J Forensic Sci, 2018, 63, 849; J Forensic Sci, 2010, 55, 953) (10,12), developed for screwdriver markings, for same‐source identification of striations on bullet LEAs. The Hamby datasets 44 and 252 measured by NIST and CSAFE (high‐resolution scans) are used here. We provide methods to identify parameters that minimize error rates for matching of LEAs, and a remedial algorithm to alleviate the problem of failed tests, while increasing the power of the test and reducing error rates. For 85,491 land‐to‐land comparisons (84,235 known nonmatches and 1256 known matches), the adapted test does not provide a result in 176 situations (originally more than 500). The Type I and Type II error rates are 7.2% (6105 out of 84,235) and 21.4% (271 out of 1256), respectively. This puts the proposed method on similar footing as other single‐feature matching approaches in the literature.
Hare, E., Hofmann, H., and Carriquiry, A. "Algorithmic Approaches to Match Degraded Land Impressions.” Law, Probability and Risk, Volume 16, 2017, pp. 203-221.
Bullet matching is a process used to determine whether two bullets may have been fired from the same gun barrel. Historically, this has been a manual process performed by trained forensic examiners. Recent work, however, has shown that it is possible to add statistical validity and objectivity to the procedure. In this article, we build upon the algorithms explored in Automatic Matching of Bullet Lands (Hare, Hofmann & Carriquiry (2017), Automatic matching of bullet lands. ArXiv E-Prints) by formalizing and defining a set of features, computed on pairs of bullet lands, which can be used in machine learning models to assess the probability of a match. We then use these features to perform an analysis of the two Hamby (Hamby, Brundage & Thorpe (2009), The identification of bullets fired from 10 consecutively rifled 9 mm Ruger pistol barrels: a research project involving 507 participants from 20 countries. AFTE J., 41, 99–110) bullet sets (Set 252 and Set 44), to assess the presence of microscope operator effects in scanning. We also take some first steps to address the issue of degraded bullet lands and provide a range of degradation at which the matching algorithm still performs well. Finally, we discuss generalizing land-to-land comparisons to full bullet comparisons as would be used for this procedure in a criminal justice situation.
Baldwin D.P., Bajic S.J., Morris M., and Zamzow D, “A study of false-positive and false-negative error rates in cartridge case comparisons.” Ames Laboratory, USDOE, Technical Report #IS-5207, 2014.
This report provides the details for a study designed to measure examiner (not laboratory) error rates for false identifications and false eliminations when comparing an unknown to a collection of three known cartridge cases. Volunteer active examiners with Association of Firearm and Toolmark Examiners (AFTE) membership or working in laboratories that participate in ASCLD were provided with 15 sets of 3 known + 1 unknown cartridge cases fired from a collection of 25 new Ruger SR9 handguns. The ammunition was all Remington 9-mm Luger (manufacturer designation L9MM3) and sets were made up of cartridge cases fired within 100 cartridges of each other for each gun. During the design phase of the experiment, examiners had expressed a concern that known samples should not be separated by a large number of fired cartridges. However, studies published on this effect indicate that several thousands of cartridges could be fired by the same firearm without making the identifying characteristics change enough to prevent identification. Examiners were provided with a background survey, an answer sheet allowing for the AFTE range of conclusions, and return shipping materials. They were also asked to assess how many of the 3 knowns were suitable for comparison, providing a measured rate of how often each firearm used in the study produces useable, quality marks. The participating examiners were provided with known positives and known negatives from independent groups of samples, providing independent measurements of a false-positive rate and independent measurements of a false-negative rate, allowing the study to measure both rates and uncertainties in those rates.
Duez, P., Weller, T., Brubaker, M., Hockensmith II, R., Lilien, R. “Development and Validation of a Virtual Examination Tool for Firearm Forensics.” Journal of Forensic Sciences, Vol 63, No 4, 2018, pp 1069-1084.
The transition from 2D imaging to 3D scanning in the discipline of firearms and toolmark analysis is likely to provide examiners an unprecedented view of microscopic surface topography. The digital examination of measured 3D surface topographies has been referred to as virtual microscopy (VM). The approach offers several potential advantages over traditional comparison microscopy. Like any new analytic method, VM must be validated prior to its use in a crime laboratory. This paper describes one of the first validation studies of virtual microscopy. Fifty‐six participants at fifteen laboratories used virtual microscopic tools to complete two proficiency‐style tests for cartridge case identification. All participating trained examiners correctly reported 100% of the identifications (known matches) while reporting no false positives. The VM tools also allowed examiners to annotate compared surfaces. These annotations provide insight into the types of marked utilized in comparative analysis. Overall, the results of the study demonstrate that trained examiners can successfully use virtual microscopy to conduct firearms toolmark examination and support the use of the technology in the crime laboratory.
Ott, D., Thompson, R., Song, J. “Applying 3D Measurements and Computer Matching Algorithms to Two Firearm Examination Proficiency Tests.” Forensic Science International, Vol. 271, 2017, pp 98-106.
In order for a crime laboratory to assess a firearms examiner’s training, skills, experience, and aptitude, it is necessary for the examiner to participate in proficiency testing. As computer algorithms for comparisons of pattern evidence become more prevalent, it is of interest to test algorithm performance as well, using these same proficiency examinations. This article demonstrates the use of the Congruent Matching Cell (CMC) algorithm to compare 3D topography measurements of breech face impressions and firing pin impressions from a previously distributed firearms proficiency test. In addition, the algorithm is used to analyze the distribution of many comparisons from a collection of cartridge cases used to construct another recent set of proficiency tests. These results are provided along with visualizations that help to relate the features used in optical comparisons by examiners to the features used by computer comparison algorithms.
Tai, X.H., Eddy, W.F., “A Fully Automatic Method for Comparing Cartridge Case Images.” Journal of Forensic Sciences, Vol. 63, No. 2, 2018, pp. 440-448.
When a gun is fired, it leaves marks on cartridge cases that are thought to be unique to the gun. In current practice, firearms examiners inspect cartridge cases for “sufficient agreement,” in which case they conclude that they come from the same gun, testifying in courts as such. A 2016 President's Council of Advisors on Science and Technology report questioned the scientific validity of such analysis (President's Committee of Advisors on Science and Technology, Washington, DC, Executive Office of the President). One recommendation was to convert firearms analysis to an objective method. We propose a fully automated, open‐source method for comparing breechface marks on cartridge cases using 2D optical images. We improve on existing methodology by automating the selection of marks, and removing the effects of circular symmetry. We propose an empirical computation of a “random match probability” given a known database, which can be used to quantify the weight of evidence. We demonstrate an improvement in accuracy on images from controlled test fires.
Chen, Z., Chu, W., Thompson, RM., Song, J., Zhao, X. “Fired bullet signature correlation using the Congruent Matching Profile Segments (CMPS) method.” Forensic Science International, 305, 2019.
We introduce the Congruent Matching Profile Segments (CMPS) method for objective comparison of striated tool marks and apply it to bullet signature correlations. The method is derived from the congruent matching cell (CMC) method developed for the comparison of impressed tool marks. The proposed method is designed to increase comparison accuracy by addressing the comparison challenges caused by striae profiles with different lateral scales, varying vertical (height) scales, and sections that are poorly marked or have little to no similarity. Instead of correlating the entire profiles extracted from striated tool marks, the method divides one of the compared profiles into segments. Each segment is then correlated with the other profile. The CMPS method uses the normalized cross-correlation function with multiple correlation peak inspection to determine the number of profile segments that have both significant topography similarity and a congruent registration position. Initial tests were performed on the land engraved areas (LEAs) of 35 bullets fired from 10 consecutively manufactured pistol barrels. The results show clear separation between the CMPS scores of the 549 known non-matching (KNM) LEA profiles and the 46 known matching (KM) LEA profiles. These results are an improvement over those obtained using the correlation coefficient score of whole profiles. The large number of CMPS segment correlations may facilitate a statistical approach to error rate estimations.
Zhang, NF. “The Use of Correlated Binomial Distribution in Estimating Error Rates for Firearms Evidence Identification.” Journal of Research of National Institute of Standards and Technology, Vol. 124, Article No. 124026, 2019.
In the branch of forensic science known as firearm evidence identification, estimating error rates is a fundamental challenge. Recently, a new quantitative approach known as the congruent matching cells (CMC) method was developed to improve the accuracy of ballistic identifications and provide a basis for estimating error rates. To estimate error rates, the key is to find an appropriate probability distribution for the relative frequency distribution of observed CMCs overlaid on a relevant measured firearm surface such as the breech face of a cartridge case. Several probability models based on the assumption of independence between cell pair comparisons have been proposed, but the assumption of independence among the cell pair comparisons from the CMC method may not be valid. This article proposes statistical models based on dependent Bernoulli trials, along with corresponding methodology for parameter estimation. To demonstrate the potential improvement from the use of the dependent Bernoulli trial model, the methodology is applied to an actual data set of fired cartridge cases.
Roberge, D., Beauchamp, A., Levesque, S. “Objective Identification of Bullets Based on 3D Pattern Matching and Line Counting Scores.” International Journal of Pattern Recognition and Artificial Intelligence, Vol. 33, No. 11, 2019.
In firearm identification, a firearm examiner looks at a pair of fired bullets or cartridge cases using a comparison microscope and determines from this visual analysis if they were both fired from the same firearm. In the particular case of fired bullets, the individual firearm signature takes the form of a striated pattern. Over the time, the firearm examiner’s community developed two distinct approaches for bullet identification: pattern matching and line counting. More recently, the emergence of technology enabling the capture of surface topographies down to a submicron depth resolution has been a catalyst for the field of computerized objective ballistic identification. Objectiveness is achieved through the statistical analysis of various scores of known matches and known nonmatches exhibit pair comparison, which in turn implies the capture of large quantities of bullets and cartridge cases topographies. The main goal of this study was to develop an objective identification method for bullets fired from conventionally rifled barrels, and to test this method on public and proprietary bullet 3D image datasets captured at different lateral resolutions. Two newly developed bullet identification scores, the Line Counting Score (LCS) and the Pattern Matching Score, computed on 3D topographies yielded perfect match versus nonmatch separation for three different sets used in the standard Hamby–Brundage Test. A similar analysis performed using a larger, more-realistic set, enabled us to define a discriminative line at a false match rate of 1/10000 on a 2D plot that shows both identification scores for matches and nonmatches. The LCS is shown to produce a better sensitivity than the standard consecutive matching striae criteria for the more-realistic dataset. A likelihood function was also computed from a linear combination of both scores, and a conservative approach based on extreme value theory is proposed to extrapolate this function in the score domain where nonmatch data are not available. This study also provides a better understanding of the limitations of studies that involve very few firearms.
van den Eden, CAJ., de Poot, CJ., van Koppen. “The Forensic Comparison Bias: A Comparison between Experts and Novices.” Journal of Forensic Sciences, Vol. 64, No. 1, 2019, pp. 120-126.
A large body of research has described the influence of context information on forensic decision‐making. In this study, we examined the effect of context information on the search for and selection of traces by students (N = 36) and crime scene investigators (N = 58). Participants investigated an ambiguous mock crime scene and received prior information indicating suicide, a violent death or no information. Participants described their impression of the scene and wrote down which traces they wanted to secure. Results showed that context information impacted first impression of the scene and crime scene behavior, namely number of traces secured. Participants in the murder condition secured most traces. Furthermore, the students secured more crime‐related traces. Students were more confident in their first impression. This study does not indicate that experts outperform novices. We therefore argue for proper training on cognitive processes as an integral part of all forensic education.
Murdock, J., Petraco, N., Thornton, J., Neel, M., Weller, T., Thompson, R., Hamby, J., Collins, E. “The Development and Application of Random Match Probabilities to Firearm and Toolmark Identification.” Journal of Forensic Sciences, Vol 62, No. 3, 2017, pp. 619-625.
The field of firearms and toolmark analysis has encountered deep scrutiny of late, stemming from a handful of voices, primarily in the law and statistical communities. While strong scrutiny is a healthy and necessary part of any scientific endeavor, much of the current criticism leveled at firearm and toolmark analysis is, at best, misinformed and, at worst, punditry. One of the most persistent criticisms stems from the view that as the field lacks quantified random match probability data (or at least a firm statistical model) with which to calculate the probability of a false match, all expert testimony concerning firearm and toolmark identification or source attribution is unreliable and should be ruled inadmissible. However, this critique does not stem from the hard work of actually obtaining data and performing the scientific research required to support or reject current findings in the literature. Although there are sound reasons (described herein) why there is currently no unifying probabilistic model for the comparison of striated and impressed toolmarks as there is in the field of forensic DNA profiling, much statistical research has been, and continues to be, done to aid the criminal justice system. This research has thus far shown that error rate estimates for the field are very low, especially when compared to other forms of judicial error. The first purpose of this paper is to point out the logical fallacies in the arguments of a small group of pundits, who advocate a particular viewpoint but cloak it as fact and research. The second purpose is to give a balanced review of the literature regarding random match probability models and statistical applications that have been carried out in forensic firearm and toolmark analysis.
Yang, M., Mou, L., Fu, Y., Wang, Y., Wang, J. “Quantitative Statistics and Identification of Tool Marks.” Journal of Forensic Sciences, Vol. 64, No. 5, 2019, pp. 1324-1334.
This study was designed to establish a feature identification method of tool‐mark 2D data. A uniform local binary pattern histogram operator was developed to extract the tool‐mark features, and the random forest algorithm was adopted to identify these. The presented method was used to conduct five groups of experiments with a 2D dataset of known matched and nonmatched tool‐marks made by bolt clippers, cutting pliers, and screwdrivers. The experimental results show that the proposed method achieved a high rate of identification of the tool‐mark samples generated under identical conditions. The proposed method effectively overcomes the disadvantage of unstable illumination of 2D tool‐mark image data and avoids the difficulty in mark inspection caused by manually preset parameters in the existing methods, thus reducing the uncertainty of inspected results.