Note
Go to the end to download the full example code.
Curation Tutorial
After spike sorting and computing quality metrics, you can automatically curate the spike sorting output using the quality metrics that you have calculated.
Import the modules and/or functions necessary from spikeinterface
import spikeinterface.core as si
Let’s generate a simulated dataset, and imagine that the ground-truth sorting is in fact the output of a sorter.
recording, sorting = si.generate_ground_truth_recording()
print(recording)
print(sorting)
GroundTruthRecording (InjectTemplatesRecording): 4 channels - 25.0kHz - 1 segments
250,000 samples - 10.00s - float32 dtype - 3.81 MiB
GroundTruthSorting (NumpySorting): 10 units - 1 segments - 25.0kHz
Create SortingAnalyzer
For this example, we will need a SortingAnalyzer and some extensions
to be computed first
analyzer = si.create_sorting_analyzer(sorting=sorting, recording=recording, format="memory")
analyzer.compute(["random_spikes", "waveforms", "templates", "noise_levels"])
analyzer.compute("principal_components", n_components=3, mode="by_channel_local")
print(analyzer)
estimate_sparsity (no parallelization): 0%| | 0/10 [00:00<?, ?it/s]
estimate_sparsity (no parallelization): 100%|██████████| 10/10 [00:00<00:00, 360.17it/s]
compute_waveforms (no parallelization): 0%| | 0/10 [00:00<?, ?it/s]
compute_waveforms (no parallelization): 100%|██████████| 10/10 [00:00<00:00, 273.87it/s]
noise_level (no parallelization): 0%| | 0/20 [00:00<?, ?it/s]
noise_level (no parallelization): 100%|██████████| 20/20 [00:00<00:00, 257.69it/s]
Fitting PCA: 0%| | 0/10 [00:00<?, ?it/s]
Fitting PCA: 100%|██████████| 10/10 [00:00<00:00, 139.44it/s]
Projecting waveforms: 0%| | 0/10 [00:00<?, ?it/s]
Projecting waveforms: 100%|██████████| 10/10 [00:00<00:00, 1301.53it/s]
SortingAnalyzer: 4 channels - 10 units - 1 segments - memory - sparse - has recording
Loaded 5 extensions: random_spikes, waveforms, templates, noise_levels, principal_components
Then we compute some quality metrics:
metrics_ext = analyzer.compute("quality_metrics", metric_names=["snr", "isi_violation", "nearest_neighbor"])
metrics = metrics_ext.get_data()
print(metrics)
snr isi_violations_ratio ... nn_hit_rate nn_miss_rate
0 25.376053 0.0 ... 0.886646 0.014349
1 14.980757 0.0 ... 0.850394 0.011127
2 28.889608 0.0 ... 0.914706 0.007593
3 20.811349 0.0 ... 0.818898 0.016691
4 9.619050 0.0 ... 0.748311 0.032434
5 8.585938 0.0 ... 0.695000 0.045438
6 2.861792 0.0 ... 0.717532 0.026720
7 30.806765 0.0 ... 0.952941 0.005000
8 15.345913 0.0 ... 0.852601 0.010579
9 5.601272 0.0 ... 0.746429 0.027174
[10 rows x 5 columns]
We can now threshold each quality metric and select units based on some rules.
The easiest and most intuitive way is to use boolean masking with a dataframe.
Then create a list of unit ids that we want to keep
keep_mask = (metrics["snr"] > 7.5) & (metrics["isi_violations_ratio"] < 0.2) & (metrics["nn_hit_rate"] > 0.80)
print(keep_mask)
keep_unit_ids = keep_mask[keep_mask].index.values
keep_unit_ids = [unit_id for unit_id in keep_unit_ids]
print(keep_unit_ids)
0 True
1 True
2 True
3 True
4 False
5 False
6 False
7 True
8 True
9 False
dtype: bool
['0', '1', '2', '3', '7', '8']
And now let’s create a sorting that contains only curated units and save it.
curated_sorting = sorting.select_units(keep_unit_ids)
print(curated_sorting)
curated_sorting.save(folder="curated_sorting", overwrite=True)
GroundTruthSorting (UnitsSelectionSorting): 6 units - 1 segments - 25.0kHz
We can also save the analyzer with only theses units
clean_analyzer = analyzer.select_units(unit_ids=keep_unit_ids, format="zarr", folder="clean_analyzer")
print(clean_analyzer)
SortingAnalyzer: 4 channels - 6 units - 1 segments - zarr - sparse - has recording
Loaded 6 extensions: random_spikes, waveforms, templates, noise_levels, principal_components, quality_metrics
Total running time of the script: (0 minutes 0.683 seconds)