Metric Readme
The DIADEM metric is implemented as a command line Java program run from a JAR file that is downloadable from this site. This requires that you have Java installed on your computer. Java is available at http://www.java.com/ where you will be directed to the appropriate version for your operating system. The metric can score one gold standard reconstruction against one test reconstruction, or several of each at one time, depending on the values of the first two parameters. Expected units are pixels for X and Y, and images for Z (the same units as those in the provided manual reconstructions). The downloadable file, DiademMetric.zip contains the following files:
The files can also be downloaded in tar format: DiademMetric.tar
The DiademMetric.jar will function on any operating system with the appropriate version of Java installed. To run the metric, unzip/extract the files from the zip file and then use the following command from the directory in which you have saved/downloaded the Diadem Metric files:
java -jar DiademMetric.jar -G [GoldStandardFilename] -T [TestReconstructionFilename]
-D [data set number]
This sample command includes the required parameters (a working example is provided at the end of this page). Required and non-required parameters are described below. Using an incorrect set of parameters will result in a description of allowable parameters and types.
Parameters:
-G / --gold-standard
Required. The gold standard reconstruction filename or directory name is required and is commonly the manual reconstruction. If a file name is entered, a file name must be entered for the test data parameter and both files must have .swc extensions. If a directory is entered, a directory must be entered for the test data parameter. Files and directories can be entered with a local (e.g. manualRecon1.swc) or global path (e.g. C:/reconstructions/manualRecon1.swc or /reconstructions/manualRecon1.swc depending on your system).
-T / --test-data
Required. The test reconstruction filename or directory name is also required and is commonly the automated reconstruction that is being tested. If a file name is entered, a file name must be entered for the gold standard parameter and both files must have .swc extensions. If a directory is entered, a directory must be entered for the gold standard parameter. Files and directories can be entered with a local (e.g. automaticRecon1.swc) or global path (e.g. C:/reconstructions/automaticRecon1.swc or /reconstructions/automaticRecon1.swc depending on your system).
-D / --dataset
Required. This parameter requires a number that identifies the data set of the reconstructions.
1. Cerebellar Climbing Fibers
2. Hippocampal CA3 Interneuron
3. Neocortical Layer 6 Axons
4. Neuromuscular Projection Fibers
5. Olfactory Projection Fibers
-m / --misses
Not required, default 'false'. Boolean parameter which when true will produce a list of node positions that are scored as misses, as well as excess nodes that appear in the automated reconstruction but not in the gold standard.
Output:
The program will output the final score according to the rules described on the Metric Description page.
Example:
Run the metric on the sample swc files included in DiademMetric.zip:
java -jar DiademMetric.jar -G ExampleGoldStandard.swc -T ExampleTest.swc -D 5 -m true
Below are the results you should see:
Score: 0.987 Nodes that were missed (position and weight): (143.56,237.16,46.733) 2 (139.21,238.6,48.066) 1 (418.35,163.83,40.641) 1 Extra nodes in test reconstruction (position and weight): (403.51,158.6,37) 1 (400.35,161.83,38.998) 1 (401.3,159.26,32.202) 1
Note: Prior to the metric version released 11/25/2009, the first missed node (143.56,237.16,46.733) was not a miss.