Query & analyze

import lamindb as ln
import lnschema_bionty as lb

lb.settings.species = "human"

💡 loaded instance: testuser1/test-flow (lamindb 0.54.1)

ln.track()

💡 notebook imports: anndata==0.9.2 lamindb==0.54.1 lnschema_bionty==0.31.2 scanpy==1.9.5

💡 Transform(id='wukchS8V976Uz8', name='Query & analyze', short_name='facs2', version='0', type=notebook, updated_at=2023-09-23 14:27:31, created_by_id='DzTjkKse')

💡 Run(id='2hiayo9fxzIjEubMXz6x', run_at=2023-09-23 14:27:31, transform_id='wukchS8V976Uz8', created_by_id='DzTjkKse')

Inspect the CellMarker registry

Inspect your aggregated cell marker registry as a DataFrame:

lb.CellMarker.filter().df().head()

	name	synonyms	gene_symbol	ncbi_gene_id	uniprotkb_id	species_id	bionty_source_id	updated_at	created_by_id
id
L0m6f7FPiDeg	CD86		CD86	942	A8K632	uHJU	pdgG	2023-09-23 14:27:11	DzTjkKse
c3dZKHFOdllB	CD33		CD33	945	P20138	uHJU	pdgG	2023-09-23 14:27:11	DzTjkKse
ljp5UfCF9HCi	TCRgd	TCRGAMMADELTA|TCRγδ	None	None	None	uHJU	pdgG	2023-09-23 14:27:11	DzTjkKse
a624IeIqbchl	CD45RA		None	None	None	uHJU	pdgG	2023-09-23 14:27:11	DzTjkKse
cFJEI6e6wml3	CD20		MS4A1	931	A0A024R507	uHJU	pdgG	2023-09-23 14:27:11	DzTjkKse

Search for a marker (synonyms aware):

lb.CellMarker.search("PD-1").head(2)

	id	synonyms	__ratio__
name
PD1	2VeZenLi2dj5	PID1|PD-1|PD 1	100.0
CD16	bspnQ0igku6c		50.0

Look up markers with auto-complete:

markers = lb.CellMarker.lookup()

markers.cd14

CellMarker(id='roEbL8zuLC5k', name='Cd14', synonyms='', gene_symbol='CD14', ncbi_gene_id='4695', uniprotkb_id='O43678', updated_at=2023-09-23 14:27:11, species_id='uHJU', bionty_source_id='pdgG', created_by_id='DzTjkKse')

Query files by markers

Query panels and datasets based on markers, e.g., which datasets have 'CD14' in the flow panel:

panels_with_cd14 = ln.FeatureSet.filter(cell_markers=markers.cd14).all()

ln.File.filter(feature_sets__in=panels_with_cd14).df()

	storage_id	key	suffix	accessor	description	version	size	hash	hash_type	transform_id	run_id	initial_version_id	updated_at	created_by_id
id
FBt7GSAUhCXpYDofIonW	h7SGIP1y	None	.h5ad	AnnData	Alpert19	None	33369696	fnzTGHE8BlkiMMIqHLDjyA	md5	OWuTtS4SAponz8	xaBM4IhwfAi1KOLU6kut	None	2023-09-23 14:27:17	DzTjkKse
eYuY9b9V1wXlA5stBxrh	h7SGIP1y	None	.h5ad	AnnData	Flow cytometry file 2	None	6837528	aWYCHE1-26gzAU6rlgoMtQ	md5	SmQmhrhigFPLz8	zD0n79CMqgtJa45Yx1NH	None	2023-09-23 14:27:26	DzTjkKse

Access registries:

features = ln.Feature.lookup()
efs = lb.ExperimentalFactor.lookup()
species = lb.Species.lookup()

Find shared cell markers between two files:

files = ln.File.filter(feature_sets__in=panels_with_cd14, species=species.human).list()
file1, file2 = files[0], files[1]

shared_markers = file1.features["var"] & file2.features["var"]
shared_markers.list("name")

['Cd14', 'CD8', 'Cd19', 'CD57', 'CD3', 'CD127', 'Cd4', 'CD28', 'CD27', 'Ccr7']

Concatenate & analyze queried files

Load files into memory and concatenate:

adata1 = file1.load()
adata2 = file2.load()

import anndata as ad

adata = ad.concat(
    [adata1, adata2],
    label="file",
    keys=[file1.description, file2.description],
)
adata

/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/anndata/_core/anndata.py:1838: UserWarning: Observation names are not unique. To make them unique, call `.obs_names_make_unique`.
  utils.warn_names_duplicates("obs")

AnnData object with n_obs × n_vars = 231130 × 10
    obs: 'file'

import scanpy as sc

sc.pp.pca(adata)
sc.pl.pca(adata, color=markers.cd14.name)

Register a concatenated dataset

If we believe that we’ll need this dataset, again, we can register a concatenated version:

dataset = ln.Dataset(adata, name="Concatenated dataset")
dataset.save()
dataset.view_flow()

# clean up test instance
!lamin delete --force test-flow
!rm -r test-flow

💡 deleting instance testuser1/test-flow
✅     deleted instance settings file: /home/runner/.lamin/instance--testuser1--test-flow.env

✅     instance cache deleted
✅     deleted '.lndb' sqlite file
❗     consider manually deleting your stored data: /home/runner/work/lamin-usecases/lamin-usecases/docs/test-flow