![](\"https://img.youtube.com/vi/H_L6LL0FRoo/0.jpg\")
\n![](\"https://github.com/MIC-DKFZ/napari-nninteractive/raw/main/imgs/nnInteractive_header.png\")
\n\n# nnInteractive: Redefining 3D Promptable Segmentation\n\nThis repository contains the napari plugin for nnInteractive. Check out the\n[python backend](https://github.com/MIC-DKFZ/nnInteractive) and [MITK integration](https://www.mitk.org/MITK-nnInteractive) for more.\n\n## What is nnInteractive?\n\n> Isensee, F.\\*, Rokuss, M.\\*, Krämer, L.\\*, Dinkelacker, S., Ravindran, A., Stritzke, F., Hamm, B., Wald, T., Langenberg, M., Ulrich, C., Deissler, J., Floca, R., & Maier-Hein, K. (2025). nnInteractive: Redefining 3D Promptable Segmentation. https://arxiv.org/abs/2503.08373 \\\n> \\*: equal contribution\n\nLink: [](https://arxiv.org/abs/2503.08373)\n\n##### Abstract:\n\nAccurate and efficient 3D segmentation is essential for both clinical and research applications.\nWhile foundation models like SAM have revolutionized interactive segmentation, their 2D design and domain shift limitations make them ill-suited for 3D medical images.\nCurrent adaptations address some of these challenges but remain limited, either lacking volumetric awareness, offering restricted interactivity, or supporting only a small set of structures and modalities.\nUsability also remains a challenge, as current tools are rarely integrated into established imaging platforms and often rely on cumbersome web-based interfaces with restricted functionality.\nWe introduce nnInteractive, the first comprehensive 3D interactive open-set segmentation method.\nIt supports diverse prompts—including points, scribbles, boxes, and a novel lasso prompt—while leveraging intuitive 2D interactions to generate full 3D segmentations.\nTrained on 120+ diverse volumetric 3D datasets (CT, MRI, PET, 3D Microscopy, etc.), nnInteractive sets a new state-of-the-art in accuracy, adaptability, and usability.\nCrucially, it is the first method integrated into widely used image viewers (e.g., Napari, MITK), ensuring broad accessibility for real-world clinical and research applications.\nExtensive benchmarking demonstrates that nnInteractive far surpasses existing methods, setting a new standard for AI-driven interactive 3D segmentation.\n\n![](\"https://github.com/MIC-DKFZ/napari-nninteractive/raw/main/imgs/figure1_method.png\")
\n\n## Demo Videos\n\n\n \n\n\n ![](\"https://img.youtube.com/vi/YoMZ7Xv7gKI/0.jpg\")
\n\n\n ![](\"https://img.youtube.com/vi/V0rqPYA3sjA/0.jpg\")
\n\n\n## Installation\n\n### Prerequisites\n\nYou need a Linux or Windows computer with a Nvidia GPU. 10GB of VRAM is recommended. Small objects should work with \\<6GB.\n\n##### 1. Create a virtual environment:\n\nnnInteractive supports Python 3.10+ and works with Conda, pip, or any other virtual environment. Here’s an example using Conda:\n\n```\nconda create -n nnInteractive python=3.12\nconda activate nnInteractive\n```\n\n##### 2. Install the correct PyTorch for your system\n\nGo to the [PyTorch homepage](https://pytorch.org/get-started/locally/) and pick the right configuration.\nNote that since recently PyTorch needs to be installed via pip. This is fine to do within your conda environment.\n\nFor Ubuntu with a Nvidia GPU, pick 'stable', 'Linux', 'Pip', 'Python', 'CUDA12.6' (if all drivers are up to date, otherwise use and older version):\n\n```\npip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126\n```\n\n##### 3. Install this repository + dependencies via\n\nInstall napari if necessary\n\n```bash\npip install napari[all]\n```\n\nInstall the plugin via pip:\n\n```bash\npip install napari-nninteractive\n```\n\nOr clone and install this repository:\n\n```bash\ngit clone https://github.com/MIC-DKFZ/napari-nninteractive\ncd napari-nninteractive\npip install -e .\n```\n\n**Note:** Model weights are automatically downloaded on first use. This can take up to a couple of minutes depending on your internet connection\n\n## Getting Started\n\nUse one of these three options to start napari and activate the plugin.\nAfterward, Drag and drop your images into napari.\n\n\\***Note if getting asked which plugin to use for opening .nii.gz files use napari-nifti.**\n\na) Start napari, then Plugins -> nnInteractive.\n\n```\nnapari\n```\n\nb) Run this to start napari with the plugin already started.\n\n```\nnapari -w napari-nninteractive\n```\n\nc) Run this to start napari with the plugin and open an image directly\n\n```\nnapari demo_data/liver_145_0000.nii.gz -w napari-nninteractive\n```\n\n# How to use\n\n**Note:** To open Nifti (.nii.gz, .nii) files we recommend to select napari-nifti.\n\n![](\"https://github.com/MIC-DKFZ/napari-nninteractive/raw/main/imgs/gui_instuctions.png\")
\n\n## Citation\n\nWhen using nnInteractive, please cite the following paper:\n\n> Isensee, F.\\*, Rokuss, M.\\*, Krämer, L.\\*, Dinkelacker, S., Ravindran, A., Stritzke, F., Hamm, B., Wald, T., Langenberg, M., Ulrich, C., Deissler, J., Floca, R., & Maier-Hein, K. (2025). nnInteractive: Redefining 3D Promptable Segmentation. https://arxiv.org/abs/2503.08373 \\\n> \\*: equal contribution\n\nLink: [](https://arxiv.org/abs/2503.08373)\n\n# License\n\nNote that while this repository is available under Apache-2.0 license (see [LICENSE](./LICENSE)), the [model checkpoint](https://huggingface.co/nnInteractive/nnInteractive) is `Creative Commons Attribution Non Commercial Share Alike 4.0`!\n\n______________________________________________________________________\n\n## Acknowledgments\n\n\n ![](\"https://github.com/MIC-DKFZ/napari-nninteractive/raw/main/imgs/Logos/HI_Logo.png\"){kind=logo}
\n ![](\"https://github.com/MIC-DKFZ/napari-nninteractive/raw/main/imgs/Logos/DKFZ_Logo.png\"){kind=logo}
\n