Show Pack Classification

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#356

.gitea/actions/setup-envipy/action.yaml

@@ -48,11 +48,6 @@ runs:
       shell: bash
       run: |
         uv run python scripts/pnpm_wrapper.py install
-        cat << 'EOF' > pnpm-workspace.yaml
-        onlyBuiltDependencies:
-          - '@parcel/watcher'
-          - '@tailwindcss/oxide'
-        EOF
         uv run python scripts/pnpm_wrapper.py run build
 
     - name: Wait for Postgres

.pre-commit-config.yaml

@@ -8,7 +8,7 @@ repos:
       - id: end-of-file-fixer
       - id: check-yaml
       - id: check-added-large-files
-        exclude: ^static/images/
+        exclude: ^static/images/|fixtures/
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.13.3

README.md

@@ -8,13 +8,12 @@ These instructions will guide you through setting up the project for local devel
 
 - Python 3.11 or later
 - [uv](https://github.com/astral-sh/uv) - Python package manager
-- **Docker and Docker Compose** - Required for running PostgreSQL database
+- **Docker and Docker Compose** - Required for running PostgreSQL database and Redis (for async Celery tasks)
 - Git
 - Make
 
 > **Note:** This application requires PostgreSQL (uses `ArrayField`). Docker is the easiest way to run PostgreSQL locally.
 
-
 ### 1. Install Dependencies
 
 This project uses `uv` to manage dependencies and `poe-the-poet` for task running. First, [install `uv` if you don't have it yet](https://docs.astral.sh/uv/guides/install-python/).
@@ -79,25 +78,48 @@ uv run poe bootstrap     # Bootstrap data only
 uv run poe shell         # Open the Django shell
 uv run poe build         # Build frontend assets and collect static files
 uv run poe clean         # Remove database volumes (WARNING: destroys all data)
+uv run poe celery        # Start Celery worker for async task processing
+uv run poe celery-dev    # Start database and Celery worker
 ```
 
+### 4. Async Celery Setup (Optional)
+
+By default, Celery tasks run synchronously (`CELERY_TASK_ALWAYS_EAGER = True`), which means prediction tasks block the HTTP request until completion. To enable asynchronous task processing with live status updates on pathway pages:
+
+1. **Set the Celery flag in your `.env` file:**
+
+   ```bash
+   FLAG_CELERY_PRESENT=True
+   ```
+
+2. **Start Redis and Celery worker:**
+
+   ```bash
+   uv run poe celery-dev
+   ```
+
+3. **Start the development server** (in another terminal):
+   ```bash
+   uv run poe dev
+   ```
+
 ### Troubleshooting
 
-*   **Docker Connection Error:** If you see an error like `open //./pipe/dockerDesktopLinuxEngine: The system cannot find the file specified` (on Windows), it likely means your Docker Desktop application is not running. Please start Docker Desktop and try the command again.
+- **Docker Connection Error:** If you see an error like `open //./pipe/dockerDesktopLinuxEngine: The system cannot find the file specified` (on Windows), it likely means your Docker Desktop application is not running. Please start Docker Desktop and try the command again.
 
-*   **SSH Keys for Git Dependencies:** Some dependencies are installed from private git repositories and require SSH authentication. Ensure your SSH keys are configured correctly for Git.
-    *   For a general guide, see [GitHub's official documentation](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent).
-    *   **Windows Users:** If `uv sync` hangs while fetching git dependencies, you may need to explicitly configure Git to use the Windows OpenSSH client and use the `ssh-agent` to manage your key's passphrase.
+- **SSH Keys for Git Dependencies:** Some dependencies are installed from private git repositories and require SSH authentication. Ensure your SSH keys are configured correctly for Git.
+  - For a general guide, see [GitHub's official documentation](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent).
+  - **Windows Users:** If `uv sync` hangs while fetching git dependencies, you may need to explicitly configure Git to use the Windows OpenSSH client and use the `ssh-agent` to manage your key's passphrase.
+    1.  **Point Git to the correct SSH executable:**
+        ```powershell
+        git config --global core.sshCommand "C:/Windows/System32/OpenSSH/ssh.exe"
+        ```
+    2.  **Enable and use the SSH agent:**
 
-        1.  **Point Git to the correct SSH executable:**
-            ```powershell
-            git config --global core.sshCommand "C:/Windows/System32/OpenSSH/ssh.exe"
-            ```
-        2.  **Enable and use the SSH agent:**
-            ```powershell
-            # Run these commands in an administrator PowerShell
-            Get-Service ssh-agent | Set-Service -StartupType Automatic -PassThru | Start-Service
+        ```powershell
+        # Run these commands in an administrator PowerShell
+        Get-Service ssh-agent | Set-Service -StartupType Automatic -PassThru | Start-Service
 
-            # Add your key to the agent. It will prompt for the passphrase once.
-            ssh-add
-            ```
+        # Add your key to the agent. It will prompt for the passphrase once.
+        ssh-add
+        ```

bayer/admin.py

@@ -0,0 +1,3 @@
+from django.contrib import admin
+
+# Register your models here.

bayer/apps.py

@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class BayerConfig(AppConfig):
+    default_auto_field = 'django.db.models.BigAutoField'
+    name = 'bayer'

bayer/migrations/0001_initial.py

@@ -0,0 +1,35 @@
+# Generated by Django 5.2.7 on 2026-03-06 10:51
+
+import django.utils.timezone
+import model_utils.fields
+import uuid
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Package',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
+                ('name', models.TextField(default='no name', verbose_name='Name')),
+                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
+                ('kv', models.JSONField(blank=True, default=dict, null=True)),
+                ('reviewed', models.BooleanField(default=False, verbose_name='Reviewstatus')),
+                ('classification_level', models.IntegerField(choices=[(0, 'Internal'), (10, 'Restricted'), (20, 'Secret')], default=10)),
+            ],
+            options={
+                'db_table': 'epdb_package',
+            },
+        ),
+    ]

bayer/migrations/0002_initial.py

@@ -0,0 +1,22 @@
+# Generated by Django 5.2.7 on 2026-03-06 10:51
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+        ('bayer', '0001_initial'),
+        ('epdb', '0019_remove_scenario_additional_information_and_more'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='package',
+            name='license',
+            field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.license', verbose_name='License'),
+        ),
+    ]

bayer/models.py

@@ -0,0 +1,95 @@
+from typing import List
+
+from django.conf import settings as s
+from django.db import models
+from django.db.models import QuerySet
+
+from epdb.models import (
+    EnviPathModel,
+    ParallelRule,
+    SequentialRule,
+    SimpleAmbitRule,
+    SimpleRDKitRule,
+)
+
+
+class Package(EnviPathModel):
+    reviewed = models.BooleanField(verbose_name="Reviewstatus", default=False)
+    license = models.ForeignKey(
+        "epdb.License", on_delete=models.SET_NULL, blank=True, null=True, verbose_name="License"
+    )
+
+    class Classification(models.IntegerChoices):
+        INTERNAL = 0, "Internal"
+        RESTRICTED = 10 , "Restricted"
+        SECRET = 20, "Secret"
+
+    classification_level = models.IntegerField(
+        choices=Classification,
+        default=Classification.RESTRICTED,
+    )
+
+    def delete(self, *args, **kwargs):
+        # explicitly handle related Rules
+        for r in self.rules.all():
+            r.delete()
+        super().delete(*args, **kwargs)
+
+    def __str__(self):
+        return f"{self.name} (pk={self.pk})"
+
+    @property
+    def compounds(self) -> QuerySet:
+        return self.compound_set.all()
+
+    @property
+    def rules(self) -> QuerySet:
+        return self.rule_set.all()
+
+    @property
+    def reactions(self) -> QuerySet:
+        return self.reaction_set.all()
+
+    @property
+    def pathways(self) -> QuerySet:
+        return self.pathway_set.all()
+
+    @property
+    def scenarios(self) -> QuerySet:
+        return self.scenario_set.all()
+
+    @property
+    def models(self) -> QuerySet:
+        return self.epmodel_set.all()
+
+    def _url(self):
+        return "{}/package/{}".format(s.SERVER_URL, self.uuid)
+
+    def get_applicable_rules(self) -> List["Rule"]:
+        """
+        Returns a ordered set of rules where the following applies:
+        1. All Composite will be added to result
+        2. All SimpleRules will be added if theres no CompositeRule present using the SimpleRule
+        Ordering is based on "url" field.
+        """
+        rules = []
+        rule_qs = self.rules
+
+        reflected_simple_rules = set()
+
+        for r in rule_qs:
+            if isinstance(r, ParallelRule) or isinstance(r, SequentialRule):
+                rules.append(r)
+                for sr in r.simple_rules.all():
+                    reflected_simple_rules.add(sr)
+
+        for r in rule_qs:
+            if isinstance(r, SimpleAmbitRule) or isinstance(r, SimpleRDKitRule):
+                if r not in reflected_simple_rules:
+                    rules.append(r)
+
+        rules = sorted(rules, key=lambda x: x.url)
+        return rules
+
+    class Meta:
+        db_table = "epdb_package"

bayer/templates/objects/package.html

@@ -0,0 +1,97 @@
+{% extends "framework_modern.html" %}
+
+{% block content %}
+
+  {% block action_modals %}
+    {% include "modals/objects/edit_package_modal.html" %}
+    {% include "modals/objects/edit_package_permissions_modal.html" %}
+    {% include "modals/objects/publish_package_modal.html" %}
+    {% include "modals/objects/set_license_modal.html" %}
+    {% include "modals/objects/export_package_modal.html" %}
+    {% include "modals/objects/generic_delete_modal.html" %}
+  {% endblock action_modals %}
+
+  <div class="space-y-2 p-4">
+    <!-- Header Section -->
+    <div class="card bg-base-100">
+      <div class="card-body">
+        <div class="flex items-center justify-between">
+          <h2 class="card-title text-2xl">{{ package.name }} - ({{ package.get_classification_level_display }})</h2>
+          <div id="actionsButton" class="dropdown dropdown-e nd hidden">
+            <div tabindex="0" role="button" class="btn btn-ghost btn-sm">
+              <svg
+                xmlns="http://www.w3.org/2000/svg"
+                width="16"
+                height="16"
+                viewBox="0 0 24 24"
+                fill="none"
+                stroke="currentColor"
+                stroke-width="2"
+                stroke-linecap="round"
+                stroke-linejoin="round"
+                class="lucide lucide-wrench"
+              >
+                <path
+                  d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"
+                />
+              </svg>
+              Actions
+            </div>
+            <ul
+              tabindex="-1"
+              class="dropdown-content menu bg-base-100 rounded-box z-50 w-52 p-2"
+            >
+              {% block actions %}
+                {% include "actions/objects/package.html" %}
+              {% endblock %}
+            </ul>
+          </div>
+        </div>
+        <p class="mt-2">{{ package.description|safe }}</p>
+        <ul class="menu bg-base-200 rounded-box mt-4 w-full">
+          <li>
+            <a href="{{ package.url }}/pathway" class="hover:bg-base-300"
+              >Pathways ({{ package.pathways.count }})</a
+            >
+          </li>
+          <li>
+            <a href="{{ package.url }}/rule" class="hover:bg-base-300"
+              >Rules ({{ package.rules.count }})</a
+            >
+          </li>
+          <li>
+            <a href="{{ package.url }}/compound" class="hover:bg-base-300"
+              >Compounds ({{ package.compounds.count }})</a
+            >
+          </li>
+          <li>
+            <a href="{{ package.url }}/reaction" class="hover:bg-base-300"
+              >Reactions ({{ package.reactions.count }})</a
+            >
+          </li>
+          <li>
+            <a href="{{ package.url }}/model" class="hover:bg-base-300"
+              >Models ({{ package.models.count }})</a
+            >
+          </li>
+          <li>
+            <a href="{{ package.url }}/scenario" class="hover:bg-base-300"
+              >Scenarios ({{ package.scenarios.count }})</a
+            >
+          </li>
+        </ul>
+      </div>
+    </div>
+  </div>
+
+  <script>
+    // Show actions button if there are actions
+    document.addEventListener("DOMContentLoaded", function () {
+      const actionsButton = document.getElementById("actionsButton");
+      const actionsList = actionsButton?.querySelector("ul");
+      if (actionsList && actionsList.children.length > 0) {
+        actionsButton?.classList.remove("hidden");
+      }
+    });
+  </script>
+{% endblock content %}

bayer/tests.py

@@ -0,0 +1,3 @@
+from django.test import TestCase
+
+# Create your tests here.

bayer/views.py

@@ -0,0 +1,3 @@
+from django.shortcuts import render
+
+# Create your views here.

bridge/contracts.py

@@ -0,0 +1,233 @@
+import enum
+from abc import ABC, abstractmethod
+
+from .dto import BuildResult, EnviPyDTO, EvaluationResult, RunResult
+
+
+class PropertyType(enum.Enum):
+    """
+    Enumeration representing different types of properties.
+
+    PropertyType is an Enum class that defines categories or types of properties
+    based on their weight or nature. It can typically be used when classifying
+    objects or entities by their weight classification, such as lightweight or heavy.
+    """
+
+    LIGHTWEIGHT = "lightweight"
+    HEAVY = "heavy"
+
+
+class Plugin(ABC):
+    """
+    Defines an abstract base class Plugin to serve as a blueprint for plugins.
+
+    This class establishes the structure that all plugin implementations must
+    follow. It enforces the presence of required methods to ensure consistent
+    functionality across all derived classes.
+
+    """
+
+    @abstractmethod
+    def identifier(self) -> str:
+        pass
+
+    @abstractmethod
+    def name(self) -> str:
+        """
+        Represents an abstract method that provides a contract for implementing a method
+        to return a name as a string. Must be implemented in subclasses.
+        Name must be unique across all plugins.
+
+        Methods
+        -------
+        name() -> str
+            Abstract method to be defined in subclasses, which returns a string
+            representing a name.
+        """
+        pass
+
+    @abstractmethod
+    def display(self) -> str:
+        """
+        An abstract method that must be implemented by subclasses to display
+        specific information or behavior. The method ensures that all subclasses
+        provide their own implementation of the display functionality.
+
+        Raises:
+            NotImplementedError: Raises this error when the method is not implemented
+            in a subclass.
+
+        Returns:
+            str: A string used in dropdown menus or other user interfaces to display
+        """
+        pass
+
+
+class Property(Plugin):
+    @abstractmethod
+    def requires_rule_packages(self) -> bool:
+        """
+        Defines an abstract method to determine whether rule packages are required.
+
+        This method should be implemented by subclasses to specify if they depend
+        on rule packages for their functioning.
+
+        Raises:
+            NotImplementedError: If the subclass has not implemented this method.
+
+        @return: A boolean indicating if rule packages are required.
+        """
+        pass
+
+    @abstractmethod
+    def requires_data_packages(self) -> bool:
+        """
+        Defines an abstract method to determine whether data packages are required.
+
+        This method should be implemented by subclasses to specify if they depend
+        on data packages for their functioning.
+
+        Raises:
+            NotImplementedError: If the subclass has not implemented this method.
+
+        Returns:
+            bool: True if the service requires data packages, False otherwise.
+        """
+        pass
+
+    @abstractmethod
+    def get_type(self) -> PropertyType:
+        """
+        An abstract method that provides the type of property. This method must
+        be implemented by subclasses to specify the appropriate property type.
+
+        Raises:
+            NotImplementedError: If the method is not implemented by a subclass.
+
+        Returns:
+            PropertyType: The type of the property associated with the implementation.
+        """
+        pass
+
+    def is_heavy(self):
+        """
+        Determines if the current property type is heavy.
+
+        This method evaluates whether the property type returned from the `get_type()`
+        method is classified as `HEAVY`. It utilizes the `PropertyType.HEAVY` constant
+        for this comparison.
+
+        Raises:
+            AttributeError: If the `get_type()` method is not defined or does not return
+            a valid value.
+
+        Returns:
+            bool: True if the property type is `HEAVY`, otherwise False.
+        """
+        return self.get_type() == PropertyType.HEAVY
+
+    @abstractmethod
+    def build(self, eP: EnviPyDTO, *args, **kwargs) -> BuildResult | None:
+        """
+        Abstract method to prepare and construct a specific build process based on the provided
+        environment data transfer object (EnviPyDTO). This method should be implemented by
+        subclasses to handle the particular requirements of the environment.
+
+        Parameters:
+        eP : EnviPyDTO
+            The data transfer object containing environment details for the build process.
+
+        *args :
+            Additional positional arguments required for the build.
+
+        **kwargs :
+            Additional keyword arguments to offer flexibility and customization for
+            the build process.
+
+        Returns:
+        BuildResult | None
+            Returns a BuildResult instance if the build operation succeeds, else returns None.
+
+        Raises:
+        NotImplementedError
+            If the method is not implemented in a subclass.
+        """
+        pass
+
+    @abstractmethod
+    def run(self, eP: EnviPyDTO, *args, **kwargs) -> RunResult:
+        """
+        Represents an abstract base class for executing a generic process with
+        provided parameters and returning a standardized result.
+
+        Attributes:
+            None.
+
+        Methods:
+            run(eP, *args, **kwargs):
+                Executes a task with specified input parameters and optional
+                arguments, returning the outcome in the form of a RunResult object.
+                This is an abstract method and must be implemented in subclasses.
+
+        Raises:
+            NotImplementedError: If the subclass does not implement the abstract
+            method.
+
+        Parameters:
+            eP (EnviPyDTO): The primary object containing information or data required
+            for processing. Mandatory.
+            *args: Variable length argument list for additional positional arguments.
+            **kwargs: Arbitrary keyword arguments for additional options or settings.
+
+        Returns:
+            RunResult: A result object encapsulating the status, output, or details
+            of the process execution.
+
+        """
+        pass
+
+    @abstractmethod
+    def evaluate(self, eP: EnviPyDTO, *args, **kwargs) -> EvaluationResult:
+        """
+        Abstract method for evaluating data based on the given input and additional arguments.
+
+        This method is intended to be implemented by subclasses and provides
+        a mechanism to perform an evaluation procedure based on input encapsulated
+        in an EnviPyDTO object.
+
+        Parameters:
+        eP : EnviPyDTO
+            The data transfer object containing necessary input for evaluation.
+        *args : tuple
+            Additional positional arguments for the evaluation process.
+        **kwargs : dict
+            Additional keyword arguments for the evaluation process.
+
+        Returns:
+        EvaluationResult
+            The result of the evaluation performed by the method.
+
+        Raises:
+        NotImplementedError
+            If the method is not implemented in the subclass.
+        """
+        pass
+
+    @abstractmethod
+    def build_and_evaluate(self, eP: EnviPyDTO, *args, **kwargs) -> EvaluationResult:
+        """
+        An abstract method designed to build and evaluate a model or system using the provided
+        environmental parameters and additional optional arguments.
+
+        Args:
+            eP (EnviPyDTO): The environmental parameters required for building and evaluating.
+            *args: Additional positional arguments.
+            **kwargs: Additional keyword arguments.
+
+        Returns:
+            EvaluationResult: The result of the evaluation process.
+
+        Raises:
+            NotImplementedError: If the method is not implemented by a subclass.
+        """
+        pass

bridge/dto.py

@@ -0,0 +1,140 @@
+from dataclasses import dataclass
+from typing import Any, List, Optional, Protocol
+
+from envipy_additional_information import EnviPyModel, register
+from pydantic import HttpUrl
+
+from utilities.chem import FormatConverter, ProductSet
+
+
+@dataclass(frozen=True, slots=True)
+class Context:
+    uuid: str
+    url: str
+    work_dir: str
+
+
+class CompoundProto(Protocol):
+    url: str | None
+    name: str | None
+    smiles: str
+
+
+class RuleProto(Protocol):
+    url: str
+    name: str
+
+    def apply(self, smiles, *args, **kwargs): ...
+
+
+class ReactionProto(Protocol):
+    url: str
+    name: str
+    rules: List[RuleProto]
+
+
+class EnviPyDTO(Protocol):
+    def get_context(self) -> Context: ...
+
+    def get_compounds(self) -> List[CompoundProto]: ...
+
+    def get_reactions(self) -> List[ReactionProto]: ...
+
+    def get_rules(self) -> List[RuleProto]: ...
+
+    @staticmethod
+    def standardize(smiles, remove_stereo=False, canonicalize_tautomers=False): ...
+
+    @staticmethod
+    def apply(
+        smiles: str,
+        smirks: str,
+        preprocess_smiles: bool = True,
+        bracketize: bool = True,
+        standardize: bool = True,
+        kekulize: bool = True,
+        remove_stereo: bool = True,
+        reactant_filter_smarts: str | None = None,
+        product_filter_smarts: str | None = None,
+    ) -> List["ProductSet"]: ...
+
+
+class PredictedProperty(EnviPyModel):
+    pass
+
+
+@register("buildresult")
+class BuildResult(EnviPyModel):
+    data: dict[str, Any] | List[dict[str, Any]] | None
+
+
+@register("runresult")
+class RunResult(EnviPyModel):
+    producer: HttpUrl
+    description: Optional[str] = None
+    result: PredictedProperty | List[PredictedProperty]
+
+
+@register("evaluationresult")
+class EvaluationResult(EnviPyModel):
+    data: dict[str, Any] | List[dict[str, Any]] | None
+
+
+class BaseDTO(EnviPyDTO):
+    def __init__(
+        self,
+        uuid: str,
+        url: str,
+        work_dir: str,
+        compounds: List[CompoundProto],
+        reactions: List[ReactionProto],
+        rules: List[RuleProto],
+    ):
+        self.uuid = uuid
+        self.url = url
+        self.work_dir = work_dir
+        self.compounds = compounds
+        self.reactions = reactions
+        self.rules = rules
+
+    def get_context(self) -> Context:
+        return Context(uuid=self.uuid, url=self.url, work_dir=self.work_dir)
+
+    def get_compounds(self) -> List[CompoundProto]:
+        return self.compounds
+
+    def get_reactions(self) -> List[ReactionProto]:
+        return self.reactions
+
+    def get_rules(self) -> List[RuleProto]:
+        return self.rules
+
+    @staticmethod
+    def standardize(smiles, remove_stereo=False, canonicalize_tautomers=False):
+        return FormatConverter.standardize(
+            smiles, remove_stereo=remove_stereo, canonicalize_tautomers=canonicalize_tautomers
+        )
+
+    @staticmethod
+    def apply(
+        smiles: str,
+        smirks: str,
+        preprocess_smiles: bool = True,
+        bracketize: bool = True,
+        standardize: bool = True,
+        kekulize: bool = True,
+        remove_stereo: bool = True,
+        reactant_filter_smarts: str | None = None,
+        product_filter_smarts: str | None = None,
+    ) -> List["ProductSet"]:
+        return FormatConverter.apply(
+            smiles,
+            smirks,
+            preprocess_smiles,
+            bracketize,
+            standardize,
+            kekulize,
+            remove_stereo,
+            reactant_filter_smarts,
+            product_filter_smarts,
+        )

docker-compose.dev.yml

@@ -1,6 +1,6 @@
 services:
   db:
-    image: postgres:15
+    image: postgres:18
     container_name: envipath-postgres
     environment:
       POSTGRES_USER: postgres
@@ -9,12 +9,18 @@ services:
     ports:
       - "5432:5432"
     volumes:
-      - postgres_data:/var/lib/postgresql/data
+      - postgres_data:/var/lib/postgresql
     healthcheck:
       test: ["CMD-SHELL", "pg_isready -U postgres"]
       interval: 5s
       timeout: 5s
       retries: 5
 
+  redis:
+    image: redis:7-alpine
+    container_name: envipath-redis
+    ports:
+      - "6379:6379"
+
 volumes:
   postgres_data:

envipath/settings.py

@@ -14,14 +14,15 @@ import os
 from pathlib import Path
 
 from dotenv import load_dotenv
-from envipy_plugins import Classifier, Property, Descriptor
 from sklearn.ensemble import RandomForestClassifier
 from sklearn.tree import DecisionTreeClassifier
 
 # Build paths inside the project like this: BASE_DIR / 'subdir'.
 BASE_DIR = Path(__file__).resolve().parent.parent
 
-load_dotenv(BASE_DIR / ".env", override=False)
+ENV_PATH = os.environ.get("ENV_PATH", BASE_DIR / ".env")
+print(f"Loading env from {ENV_PATH}")
+load_dotenv(ENV_PATH, override=False)
 
 # Quick-start development settings - unsuitable for production
 # See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/
@@ -50,7 +51,7 @@ INSTALLED_APPS = [
     # Custom
     "epapi",  # API endpoints (v1, etc.)
     "epdb",
-    # "migration",
+    "migration",
 ]
 
 TENANT = os.environ.get("TENANT", "public")
@@ -91,10 +92,19 @@ if os.environ.get("REGISTRATION_MANDATORY", False) == "True":
 
 ROOT_URLCONF = "envipath.urls"
 
+TEMPLATE_DIRS = [
+    os.path.join(BASE_DIR, "templates"),
+]
+
+# If we have a non-public tenant, we might need to overwrite some templates
+# search TENANT folder first...
+if TENANT != "public":
+    TEMPLATE_DIRS.insert(0, os.path.join(BASE_DIR, TENANT, "templates"))
+
 TEMPLATES = [
     {
         "BACKEND": "django.template.backends.django.DjangoTemplates",
-        "DIRS": (os.path.join(BASE_DIR, "templates"),),
+        "DIRS": TEMPLATE_DIRS,
         "APP_DIRS": True,
         "OPTIONS": {
             "context_processors": [
@@ -126,6 +136,13 @@ DATABASES = {
     }
 }
 
+if os.environ.get("USE_TEMPLATE_DB", False) == "True":
+    DATABASES["default"]["TEST"] = {
+        "NAME": f"test_{os.environ['TEMPLATE_DB']}",
+        "TEMPLATE": os.environ["TEMPLATE_DB"],
+    }
+
+
 # Password validation
 # https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators
 
@@ -309,22 +326,19 @@ DEFAULT_MODEL_PARAMS = {
     "num_chains": 10,
 }
 
-DEFAULT_MAX_NUMBER_OF_NODES = 30
-DEFAULT_MAX_DEPTH = 5
+DEFAULT_MAX_NUMBER_OF_NODES = 50
+DEFAULT_MAX_DEPTH = 8
 DEFAULT_MODEL_THRESHOLD = 0.25
 
 # Loading Plugins
 PLUGINS_ENABLED = os.environ.get("PLUGINS_ENABLED", "False") == "True"
-if PLUGINS_ENABLED:
-    from utilities.plugin import discover_plugins
+BASE_PLUGINS = [
+    "pepper.PEPPER",
+]
 
-    CLASSIFIER_PLUGINS = discover_plugins(_cls=Classifier)
-    PROPERTY_PLUGINS = discover_plugins(_cls=Property)
-    DESCRIPTOR_PLUGINS = discover_plugins(_cls=Descriptor)
-else:
-    CLASSIFIER_PLUGINS = {}
-    PROPERTY_PLUGINS = {}
-    DESCRIPTOR_PLUGINS = {}
+CLASSIFIER_PLUGINS = {}
+PROPERTY_PLUGINS = {}
+DESCRIPTOR_PLUGINS = {}
 
 SENTRY_ENABLED = os.environ.get("SENTRY_ENABLED", "False") == "True"
 if SENTRY_ENABLED:

epapi/tests/utils/__init__.py

@@ -0,0 +1 @@
+"""Tests for epapi utility modules."""

epapi/tests/utils/test_validation_errors.py

@@ -0,0 +1,218 @@
+"""
+Tests for validation error utilities.
+
+Tests the format_validation_error() and handle_validation_error() functions
+that transform Pydantic validation errors into user-friendly messages.
+"""
+
+from django.test import TestCase, tag
+import json
+from pydantic import BaseModel, ValidationError, field_validator
+from typing import Literal
+
+from ninja.errors import HttpError
+from epapi.utils.validation_errors import format_validation_error, handle_validation_error
+
+
+@tag("api", "utils")
+class ValidationErrorUtilityTests(TestCase):
+    """Test validation error utility functions."""
+
+    def test_format_missing_field_error(self):
+        """Test formatting of missing required field error."""
+
+        # Create a model with required field
+        class TestModel(BaseModel):
+            required_field: str
+
+        # Trigger validation error
+        try:
+            TestModel()
+        except ValidationError as e:
+            errors = e.errors()
+            self.assertEqual(len(errors), 1)
+            formatted = format_validation_error(errors[0])
+            self.assertEqual(formatted, "This field is required")
+
+    def test_format_enum_error(self):
+        """Test formatting of enum validation error."""
+
+        class TestModel(BaseModel):
+            status: Literal["active", "inactive"]
+
+        try:
+            TestModel(status="invalid")
+        except ValidationError as e:
+            errors = e.errors()
+            self.assertEqual(len(errors), 1)
+            formatted = format_validation_error(errors[0])
+            # Literal errors get formatted as "Please enter ..." with the valid options
+            self.assertIn("Please enter", formatted)
+            self.assertIn("active", formatted)
+            self.assertIn("inactive", formatted)
+
+    def test_format_type_errors(self):
+        """Test formatting of type validation errors (string, int, float)."""
+        test_cases = [
+            # (field_type, invalid_value, expected_message)
+            # Note: We don't check exact error_type as Pydantic may use different types
+            # (e.g., int_type vs int_parsing) but we verify the formatted message is correct
+            (str, 123, "Please enter a valid string"),
+            (int, "not_a_number", "Please enter a valid int"),
+            (float, "not_a_float", "Please enter a valid float"),
+        ]
+
+        for field_type, invalid_value, expected_message in test_cases:
+            with self.subTest(field_type=field_type.__name__):
+
+                class TestModel(BaseModel):
+                    field: field_type
+
+                try:
+                    TestModel(field=invalid_value)
+                except ValidationError as e:
+                    errors = e.errors()
+                    self.assertEqual(len(errors), 1)
+                    formatted = format_validation_error(errors[0])
+                    self.assertEqual(formatted, expected_message)
+
+    def test_format_value_error(self):
+        """Test formatting of value error from custom validator."""
+
+        class TestModel(BaseModel):
+            age: int
+
+            @field_validator("age")
+            @classmethod
+            def validate_age(cls, v):
+                if v < 0:
+                    raise ValueError("Age must be positive")
+                return v
+
+        try:
+            TestModel(age=-5)
+        except ValidationError as e:
+            errors = e.errors()
+            self.assertEqual(len(errors), 1)
+            formatted = format_validation_error(errors[0])
+            self.assertEqual(formatted, "Age must be positive")
+
+    def test_format_unknown_error_type_fallback(self):
+        """Test that unknown error types fall back to default formatting."""
+        # Mock an error with an unknown type
+        mock_error = {
+            "type": "unknown_custom_type",
+            "msg": "Input should be a valid email address",
+            "ctx": {},
+        }
+
+        formatted = format_validation_error(mock_error)
+        # Should use the else branch which does replacements on the message
+        self.assertEqual(formatted, "Please enter a valid email address")
+
+    def test_handle_validation_error_structure(self):
+        """Test that handle_validation_error raises HttpError with correct structure."""
+
+        class TestModel(BaseModel):
+            name: str
+            count: int
+
+        try:
+            TestModel(name=123, count="invalid")
+        except ValidationError as e:
+            # handle_validation_error should raise HttpError
+            with self.assertRaises(HttpError) as context:
+                handle_validation_error(e)
+
+            http_error = context.exception
+            self.assertEqual(http_error.status_code, 400)
+
+            # Parse the JSON from the error message
+            error_data = json.loads(http_error.message)
+
+            # Check structure
+            self.assertEqual(error_data["type"], "validation_error")
+            self.assertIn("field_errors", error_data)
+            self.assertIn("message", error_data)
+            self.assertEqual(error_data["message"], "Please correct the errors below")
+
+            # Check that both fields have errors
+            self.assertIn("name", error_data["field_errors"])
+            self.assertIn("count", error_data["field_errors"])
+
+    def test_handle_validation_error_no_pydantic_internals(self):
+        """Test that handle_validation_error doesn't expose Pydantic internals."""
+
+        class TestModel(BaseModel):
+            email: str
+
+        try:
+            TestModel(email=123)
+        except ValidationError as e:
+            with self.assertRaises(HttpError) as context:
+                handle_validation_error(e)
+
+            http_error = context.exception
+            error_data = json.loads(http_error.message)
+            error_str = json.dumps(error_data)
+
+            # Ensure no Pydantic internals are exposed
+            self.assertNotIn("pydantic", error_str.lower())
+            self.assertNotIn("https://errors.pydantic.dev", error_str)
+            self.assertNotIn("loc", error_str)
+
+    def test_handle_validation_error_user_friendly_messages(self):
+        """Test that all error messages are user-friendly."""
+
+        class TestModel(BaseModel):
+            name: str
+            age: int
+            status: Literal["active", "inactive"]
+
+        try:
+            TestModel(name=123, status="invalid")  # Multiple errors
+        except ValidationError as e:
+            with self.assertRaises(HttpError) as context:
+                handle_validation_error(e)
+
+            http_error = context.exception
+            error_data = json.loads(http_error.message)
+
+            # All messages should be user-friendly (contain "Please" or "This field")
+            for field, messages in error_data["field_errors"].items():
+                for message in messages:
+                    # User-friendly messages start with "Please" or "This field"
+                    self.assertTrue(
+                        message.startswith("Please") or message.startswith("This field"),
+                        f"Message '{message}' is not user-friendly",
+                    )
+
+    def test_handle_validation_error_multiple_errors_same_field(self):
+        """Test handling multiple validation errors for the same field."""
+
+        class TestModel(BaseModel):
+            value: int
+
+            @field_validator("value")
+            @classmethod
+            def validate_range(cls, v):
+                if v < 0:
+                    raise ValueError("Must be non-negative")
+                if v > 100:
+                    raise ValueError("Must be at most 100")
+                return v
+
+        # Test with string (type error) - this will fail before the validator runs
+        try:
+            TestModel(value="invalid")
+        except ValidationError as e:
+            with self.assertRaises(HttpError) as context:
+                handle_validation_error(e)
+
+            http_error = context.exception
+            error_data = json.loads(http_error.message)
+
+            # Should have error for 'value' field
+            self.assertIn("value", error_data["field_errors"])
+            self.assertIsInstance(error_data["field_errors"]["value"], list)
+            self.assertGreater(len(error_data["field_errors"]["value"]), 0)

epapi/tests/v1/test_additional_information.py

@@ -0,0 +1,446 @@
+"""
+Tests for Additional Information API endpoints.
+
+Tests CRUD operations on scenario additional information including the new PATCH endpoint.
+"""
+
+from django.test import TestCase, tag
+import json
+from uuid import uuid4
+
+from epdb.logic import PackageManager, UserManager
+from epdb.models import Scenario
+
+
+@tag("api", "additional_information")
+class AdditionalInformationAPITests(TestCase):
+    """Test additional information API endpoints."""
+
+    @classmethod
+    def setUpTestData(cls):
+        """Set up test data: user, package, and scenario."""
+        cls.user = UserManager.create_user(
+            "ai-test-user",
+            "ai-test@envipath.com",
+            "SuperSafe",
+            set_setting=False,
+            add_to_group=False,
+            is_active=True,
+        )
+        cls.other_user = UserManager.create_user(
+            "ai-other-user",
+            "ai-other@envipath.com",
+            "SuperSafe",
+            set_setting=False,
+            add_to_group=False,
+            is_active=True,
+        )
+        cls.package = PackageManager.create_package(
+            cls.user, "AI Test Package", "Test package for additional information"
+        )
+        # Package owned by other_user (no access for cls.user)
+        cls.other_package = PackageManager.create_package(
+            cls.other_user, "Other Package", "Package without access"
+        )
+        # Create a scenario for testing
+        cls.scenario = Scenario.objects.create(
+            package=cls.package,
+            name="Test Scenario",
+            description="Test scenario for additional information tests",
+            scenario_type="biodegradation",
+            scenario_date="2024-01-01",
+        )
+        cls.other_scenario = Scenario.objects.create(
+            package=cls.other_package,
+            name="Other Scenario",
+            description="Scenario in package without access",
+            scenario_type="biodegradation",
+            scenario_date="2024-01-01",
+        )
+
+    def test_list_all_schemas(self):
+        """Test GET /api/v1/information/schema/ returns all schemas."""
+        self.client.force_login(self.user)
+
+        response = self.client.get("/api/v1/information/schema/")
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertIsInstance(data, dict)
+        # Should have multiple schemas
+        self.assertGreater(len(data), 0)
+        # Each schema should have RJSF format
+        for name, schema in data.items():
+            self.assertIn("schema", schema)
+            self.assertIn("uiSchema", schema)
+            self.assertIn("formData", schema)
+            self.assertIn("groups", schema)
+
+    def test_get_specific_schema(self):
+        """Test GET /api/v1/information/schema/{model_name}/ returns specific schema."""
+        self.client.force_login(self.user)
+
+        # Assuming 'temperature' is a valid model
+        response = self.client.get("/api/v1/information/schema/temperature/")
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertIn("schema", data)
+        self.assertIn("uiSchema", data)
+
+    def test_get_nonexistent_schema_returns_404(self):
+        """Test GET for non-existent schema returns 404."""
+        self.client.force_login(self.user)
+
+        response = self.client.get("/api/v1/information/schema/nonexistent/")
+
+        self.assertEqual(response.status_code, 404)
+
+    def test_list_scenario_information_empty(self):
+        """Test GET /api/v1/scenario/{uuid}/information/ returns empty list initially."""
+        self.client.force_login(self.user)
+
+        response = self.client.get(f"/api/v1/scenario/{self.scenario.uuid}/information/")
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertIsInstance(data, list)
+        self.assertEqual(len(data), 0)
+
+    def test_create_additional_information(self):
+        """Test POST creates additional information."""
+        self.client.force_login(self.user)
+
+        # Create temperature information (assuming temperature model exists)
+        payload = {"interval": {"start": 20, "end": 25}}
+        response = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(data["status"], "created")
+        self.assertIn("uuid", data)
+        self.assertIsNotNone(data["uuid"])
+
+    def test_create_with_invalid_data_returns_400(self):
+        """Test POST with invalid data returns 400 with validation errors."""
+        self.client.force_login(self.user)
+
+        # Invalid data (missing required fields or wrong types)
+        payload = {"invalid_field": "value"}
+        response = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 400)
+        data = response.json()
+        # Should have validation error details in 'detail' field
+        self.assertIn("detail", data)
+
+    def test_validation_errors_are_user_friendly(self):
+        """Test that validation errors are user-friendly and field-specific."""
+        self.client.force_login(self.user)
+
+        # Invalid data - wrong type (string instead of number in interval)
+        payload = {"interval": {"start": "not_a_number", "end": 25}}
+        response = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 400)
+        data = response.json()
+
+        # Parse the error response - Django Ninja wraps errors in 'detail'
+        error_str = data.get("detail") or data.get("error")
+        self.assertIsNotNone(error_str, "Response should contain error details")
+
+        # Parse the JSON error string
+        error_data = json.loads(error_str)
+
+        # Check structure
+        self.assertEqual(error_data.get("type"), "validation_error")
+        self.assertIn("field_errors", error_data)
+        self.assertIn("message", error_data)
+
+        # Ensure error messages are user-friendly (no Pydantic URLs or technical jargon)
+        error_str = json.dumps(error_data)
+        self.assertNotIn("pydantic", error_str.lower())
+        self.assertNotIn("https://errors.pydantic.dev", error_str)
+        self.assertNotIn("loc", error_str)  # No technical field like 'loc'
+
+        # Check that error message is helpful
+        self.assertIn("Please", error_data["message"])  # User-friendly language
+
+    def test_patch_additional_information(self):
+        """Test PATCH updates existing additional information."""
+        self.client.force_login(self.user)
+
+        # First create an item
+        create_payload = {"interval": {"start": 20, "end": 25}}
+        create_response = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(create_payload),
+            content_type="application/json",
+        )
+        item_uuid = create_response.json()["uuid"]
+
+        # Then update it with PATCH
+        update_payload = {"interval": {"start": 30, "end": 35}}
+        patch_response = self.client.patch(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/item/{item_uuid}/",
+            data=json.dumps(update_payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(patch_response.status_code, 200)
+        data = patch_response.json()
+        self.assertEqual(data["status"], "updated")
+        self.assertEqual(data["uuid"], item_uuid)  # UUID preserved
+
+        # Verify the data was updated
+        list_response = self.client.get(f"/api/v1/scenario/{self.scenario.uuid}/information/")
+        items = list_response.json()
+        self.assertEqual(len(items), 1)
+        updated_item = items[0]
+        self.assertEqual(updated_item["uuid"], item_uuid)
+        self.assertEqual(updated_item["data"]["interval"]["start"], 30)
+        self.assertEqual(updated_item["data"]["interval"]["end"], 35)
+
+    def test_patch_nonexistent_item_returns_404(self):
+        """Test PATCH on non-existent item returns 404."""
+        self.client.force_login(self.user)
+
+        fake_uuid = str(uuid4())
+        payload = {"interval": {"start": 30, "end": 35}}
+        response = self.client.patch(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/item/{fake_uuid}/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 404)
+
+    def test_patch_with_invalid_data_returns_400(self):
+        """Test PATCH with invalid data returns 400."""
+        self.client.force_login(self.user)
+
+        # First create an item
+        create_payload = {"interval": {"start": 20, "end": 25}}
+        create_response = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(create_payload),
+            content_type="application/json",
+        )
+        item_uuid = create_response.json()["uuid"]
+
+        # Try to update with invalid data
+        invalid_payload = {"invalid_field": "value"}
+        patch_response = self.client.patch(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/item/{item_uuid}/",
+            data=json.dumps(invalid_payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(patch_response.status_code, 400)
+
+    def test_patch_validation_errors_are_user_friendly(self):
+        """Test that PATCH validation errors are user-friendly and field-specific."""
+        self.client.force_login(self.user)
+
+        # First create an item
+        create_payload = {"interval": {"start": 20, "end": 25}}
+        create_response = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(create_payload),
+            content_type="application/json",
+        )
+        item_uuid = create_response.json()["uuid"]
+
+        # Update with invalid data - wrong type (string instead of number in interval)
+        invalid_payload = {"interval": {"start": "not_a_number", "end": 25}}
+        patch_response = self.client.patch(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/item/{item_uuid}/",
+            data=json.dumps(invalid_payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(patch_response.status_code, 400)
+        data = patch_response.json()
+
+        # Parse the error response - Django Ninja wraps errors in 'detail'
+        error_str = data.get("detail") or data.get("error")
+        self.assertIsNotNone(error_str, "Response should contain error details")
+
+        # Parse the JSON error string
+        error_data = json.loads(error_str)
+
+        # Check structure
+        self.assertEqual(error_data.get("type"), "validation_error")
+        self.assertIn("field_errors", error_data)
+        self.assertIn("message", error_data)
+
+        # Ensure error messages are user-friendly (no Pydantic URLs or technical jargon)
+        error_str = json.dumps(error_data)
+        self.assertNotIn("pydantic", error_str.lower())
+        self.assertNotIn("https://errors.pydantic.dev", error_str)
+        self.assertNotIn("loc", error_str)  # No technical field like 'loc'
+
+        # Check that error message is helpful
+        self.assertIn("Please", error_data["message"])  # User-friendly language
+
+    def test_delete_additional_information(self):
+        """Test DELETE removes additional information."""
+        self.client.force_login(self.user)
+
+        # Create an item
+        create_payload = {"interval": {"start": 20, "end": 25}}
+        create_response = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(create_payload),
+            content_type="application/json",
+        )
+        item_uuid = create_response.json()["uuid"]
+
+        # Delete it
+        delete_response = self.client.delete(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/item/{item_uuid}/"
+        )
+
+        self.assertEqual(delete_response.status_code, 200)
+        data = delete_response.json()
+        self.assertEqual(data["status"], "deleted")
+
+        # Verify deletion
+        list_response = self.client.get(f"/api/v1/scenario/{self.scenario.uuid}/information/")
+        items = list_response.json()
+        self.assertEqual(len(items), 0)
+
+    def test_delete_nonexistent_item_returns_404(self):
+        """Test DELETE on non-existent item returns 404."""
+        self.client.force_login(self.user)
+
+        fake_uuid = str(uuid4())
+        response = self.client.delete(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/item/{fake_uuid}/"
+        )
+
+        self.assertEqual(response.status_code, 404)
+
+    def test_multiple_items_crud(self):
+        """Test creating, updating, and deleting multiple items."""
+        self.client.force_login(self.user)
+
+        # Create first item
+        item1_payload = {"interval": {"start": 20, "end": 25}}
+        response1 = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(item1_payload),
+            content_type="application/json",
+        )
+        item1_uuid = response1.json()["uuid"]
+
+        # Create second item (different type if available, or same type)
+        item2_payload = {"interval": {"start": 30, "end": 35}}
+        response2 = self.client.post(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/temperature/",
+            data=json.dumps(item2_payload),
+            content_type="application/json",
+        )
+        item2_uuid = response2.json()["uuid"]
+
+        # Verify both exist
+        list_response = self.client.get(f"/api/v1/scenario/{self.scenario.uuid}/information/")
+        items = list_response.json()
+        self.assertEqual(len(items), 2)
+
+        # Update first item
+        update_payload = {"interval": {"start": 15, "end": 20}}
+        self.client.patch(
+            f"/api/v1/scenario/{self.scenario.uuid}/information/item/{item1_uuid}/",
+            data=json.dumps(update_payload),
+            content_type="application/json",
+        )
+
+        # Delete second item
+        self.client.delete(f"/api/v1/scenario/{self.scenario.uuid}/information/item/{item2_uuid}/")
+
+        # Verify final state: one item with updated data
+        list_response = self.client.get(f"/api/v1/scenario/{self.scenario.uuid}/information/")
+        items = list_response.json()
+        self.assertEqual(len(items), 1)
+        self.assertEqual(items[0]["uuid"], item1_uuid)
+        self.assertEqual(items[0]["data"]["interval"]["start"], 15)
+
+    def test_list_info_denied_without_permission(self):
+        """User cannot list info for scenario in package they don't have access to"""
+        self.client.force_login(self.user)
+        response = self.client.get(f"/api/v1/scenario/{self.other_scenario.uuid}/information/")
+        self.assertEqual(response.status_code, 403)
+
+    def test_add_info_denied_without_permission(self):
+        """User cannot add info to scenario in package they don't have access to"""
+        self.client.force_login(self.user)
+        payload = {"interval": {"start": 25, "end": 30}}
+        response = self.client.post(
+            f"/api/v1/scenario/{self.other_scenario.uuid}/information/temperature/",
+            json.dumps(payload),
+            content_type="application/json",
+        )
+        self.assertEqual(response.status_code, 403)
+
+    def test_update_info_denied_without_permission(self):
+        """User cannot update info in scenario they don't have access to"""
+        self.client.force_login(self.other_user)
+        # First create an item as other_user
+        create_payload = {"interval": {"start": 20, "end": 25}}
+        create_response = self.client.post(
+            f"/api/v1/scenario/{self.other_scenario.uuid}/information/temperature/",
+            data=json.dumps(create_payload),
+            content_type="application/json",
+        )
+        item_uuid = create_response.json()["uuid"]
+
+        # Try to update as user (who doesn't have access)
+        self.client.force_login(self.user)
+        update_payload = {"interval": {"start": 30, "end": 35}}
+        response = self.client.patch(
+            f"/api/v1/scenario/{self.other_scenario.uuid}/information/item/{item_uuid}/",
+            data=json.dumps(update_payload),
+            content_type="application/json",
+        )
+        self.assertEqual(response.status_code, 403)
+
+    def test_delete_info_denied_without_permission(self):
+        """User cannot delete info from scenario they don't have access to"""
+        self.client.force_login(self.other_user)
+        # First create an item as other_user
+        create_payload = {"interval": {"start": 20, "end": 25}}
+        create_response = self.client.post(
+            f"/api/v1/scenario/{self.other_scenario.uuid}/information/temperature/",
+            data=json.dumps(create_payload),
+            content_type="application/json",
+        )
+        item_uuid = create_response.json()["uuid"]
+
+        # Try to delete as user (who doesn't have access)
+        self.client.force_login(self.user)
+        response = self.client.delete(
+            f"/api/v1/scenario/{self.other_scenario.uuid}/information/item/{item_uuid}/"
+        )
+        self.assertEqual(response.status_code, 403)
+
+    def test_nonexistent_scenario_returns_404(self):
+        """Test operations on non-existent scenario return 404."""
+        self.client.force_login(self.user)
+
+        fake_uuid = uuid4()
+        response = self.client.get(f"/api/v1/scenario/{fake_uuid}/information/")
+
+        self.assertEqual(response.status_code, 404)

epapi/tests/v1/test_api_permissions.py

@@ -261,13 +261,6 @@ class GlobalCompoundListPermissionTest(APIPermissionTestBase):
         self.assertEqual(response.status_code, 200)
         payload = response.json()
 
-        # user2 should see compounds from:
-        # - reviewed_package (public)
-        # - unreviewed_package_read (READ permission)
-        # - unreviewed_package_write (WRITE permission)
-        # - unreviewed_package_all (ALL permission)
-        # - group_package (via group membership)
-        # Total: 5 compounds
         self.assertEqual(payload["total_items"], 5)
 
         visible_uuids = {item["uuid"] for item in payload["items"]}
@@ -303,54 +296,6 @@ class GlobalCompoundListPermissionTest(APIPermissionTestBase):
         # user1 owns all packages, so sees all compounds
         self.assertEqual(payload["total_items"], 7)
 
-    def test_read_permission_allows_viewing(self):
-        """READ permission allows viewing compounds."""
-        self.client.force_login(self.user2)
-        response = self.client.get(self.ENDPOINT)
-
-        self.assertEqual(response.status_code, 200)
-        payload = response.json()
-
-        # Check that read_compound is included
-        uuids = [item["uuid"] for item in payload["items"]]
-        self.assertIn(str(self.read_compound.uuid), uuids)
-
-    def test_write_permission_allows_viewing(self):
-        """WRITE permission also allows viewing compounds."""
-        self.client.force_login(self.user2)
-        response = self.client.get(self.ENDPOINT)
-
-        self.assertEqual(response.status_code, 200)
-        payload = response.json()
-
-        # Check that write_compound is included
-        uuids = [item["uuid"] for item in payload["items"]]
-        self.assertIn(str(self.write_compound.uuid), uuids)
-
-    def test_all_permission_allows_viewing(self):
-        """ALL permission allows viewing compounds."""
-        self.client.force_login(self.user2)
-        response = self.client.get(self.ENDPOINT)
-
-        self.assertEqual(response.status_code, 200)
-        payload = response.json()
-
-        # Check that all_compound is included
-        uuids = [item["uuid"] for item in payload["items"]]
-        self.assertIn(str(self.all_compound.uuid), uuids)
-
-    def test_group_permission_allows_viewing(self):
-        """Group membership grants access to group-permitted packages."""
-        self.client.force_login(self.user2)
-        response = self.client.get(self.ENDPOINT)
-
-        self.assertEqual(response.status_code, 200)
-        payload = response.json()
-
-        # Check that group_compound is included
-        uuids = [item["uuid"] for item in payload["items"]]
-        self.assertIn(str(self.group_compound.uuid), uuids)
-
 
 @tag("api", "end2end")
 class PackageScopedCompoundListPermissionTest(APIPermissionTestBase):

epapi/tests/v1/test_contract_get_entities.py

@@ -134,7 +134,7 @@ class BaseTestAPIGetPaginated:
                 f"({self.total_reviewed} <= {self.default_page_size})"
             )
 
-        response = self.client.get(self.global_endpoint, {"page": 2})
+        response = self.client.get(self.global_endpoint, {"page": 2, "review_status": True})
         self.assertEqual(response.status_code, 200)
 
         payload = response.json()

epapi/tests/v1/test_scenario_creation.py

@@ -0,0 +1,301 @@
+"""
+Tests for Scenario Creation Endpoint Error Handling.
+
+Tests comprehensive error handling for POST /api/v1/package/{uuid}/scenario/
+including package not found, permission denied, validation errors, and database errors.
+"""
+
+from django.test import TestCase, tag
+import json
+from uuid import uuid4
+
+from epdb.logic import PackageManager, UserManager
+from epdb.models import Scenario
+
+
+@tag("api", "scenario_creation")
+class ScenarioCreationAPITests(TestCase):
+    """Test scenario creation endpoint error handling."""
+
+    @classmethod
+    def setUpTestData(cls):
+        """Set up test data: users and packages."""
+        cls.user = UserManager.create_user(
+            "scenario-test-user",
+            "scenario-test@envipath.com",
+            "SuperSafe",
+            set_setting=False,
+            add_to_group=False,
+            is_active=True,
+        )
+        cls.other_user = UserManager.create_user(
+            "other-user",
+            "other@envipath.com",
+            "SuperSafe",
+            set_setting=False,
+            add_to_group=False,
+            is_active=True,
+        )
+        cls.package = PackageManager.create_package(
+            cls.user, "Test Package", "Test package for scenario creation"
+        )
+
+    def test_create_scenario_package_not_found(self):
+        """Test that non-existent package UUID returns 404."""
+        self.client.force_login(self.user)
+
+        fake_uuid = uuid4()
+        payload = {
+            "name": "Test Scenario",
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{fake_uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 404)
+        self.assertIn(f"Package with UUID {fake_uuid} not found", response.json()["detail"])
+
+    def test_create_scenario_insufficient_permissions(self):
+        """Test that unauthorized access returns 403."""
+        self.client.force_login(self.other_user)
+
+        payload = {
+            "name": "Test Scenario",
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 403)
+        self.assertIn("permission", response.json()["detail"].lower())
+
+    def test_create_scenario_invalid_ai_type(self):
+        """Test that unknown additional information type returns 400."""
+        self.client.force_login(self.user)
+
+        payload = {
+            "name": "Test Scenario",
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [
+                {"type": "invalid_type_that_does_not_exist", "data": {"some_field": "some_value"}}
+            ],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 400)
+        response_data = response.json()
+        self.assertIn("Validation errors", response_data["detail"])
+
+    def test_create_scenario_validation_error(self):
+        """Test that invalid additional information data returns 400."""
+        self.client.force_login(self.user)
+
+        # Use malformed data structure for an actual AI type
+        payload = {
+            "name": "Test Scenario",
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [
+                {
+                    "type": "invalid_type_name",
+                    "data": None,  # This should cause a validation error
+                }
+            ],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        # Should return 422 for validation errors
+        self.assertEqual(response.status_code, 422)
+
+    def test_create_scenario_success(self):
+        """Test that valid scenario creation returns 200."""
+        self.client.force_login(self.user)
+
+        payload = {
+            "name": "Test Scenario",
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(data["name"], "Test Scenario")
+        self.assertEqual(data["description"], "Test description")
+
+        # Verify scenario was actually created
+        scenario = Scenario.objects.get(name="Test Scenario")
+        self.assertEqual(scenario.package, self.package)
+        self.assertEqual(scenario.scenario_type, "biodegradation")
+
+    def test_create_scenario_auto_name(self):
+        """Test that empty name triggers auto-generation."""
+        self.client.force_login(self.user)
+
+        payload = {
+            "name": "",  # Empty name should be auto-generated
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        # Auto-generated name should follow pattern "Scenario N"
+        self.assertTrue(data["name"].startswith("Scenario "))
+
+    def test_create_scenario_xss_protection(self):
+        """Test that XSS attempts are sanitized."""
+        self.client.force_login(self.user)
+
+        payload = {
+            "name": "<script>alert('xss')</script>Clean Name",
+            "description": "<img src=x onerror=alert('xss')>Description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        # XSS should be cleaned out
+        self.assertNotIn("<script>", data["name"])
+        self.assertNotIn("onerror", data["description"])
+
+    def test_create_scenario_missing_required_field(self):
+        """Test that missing required fields returns validation error."""
+        self.client.force_login(self.user)
+
+        # Missing 'name' field entirely
+        payload = {
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        # Should return 422 for schema validation errors
+        self.assertEqual(response.status_code, 422)
+
+    def test_create_scenario_type_error_in_ai(self):
+        """Test that TypeError in AI instantiation returns 400."""
+        self.client.force_login(self.user)
+
+        payload = {
+            "name": "Test Scenario",
+            "description": "Test description",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [
+                {
+                    "type": "invalid_type_name",
+                    "data": "string instead of dict",  # Wrong type
+                }
+            ],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        # Should return 422 for validation errors
+        self.assertEqual(response.status_code, 422)
+
+    def test_create_scenario_default_values(self):
+        """Test that default values are applied correctly."""
+        self.client.force_login(self.user)
+
+        # Minimal payload with only name
+        payload = {"name": "Minimal Scenario"}
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertEqual(data["name"], "Minimal Scenario")
+        # Check defaults are applied
+        scenario = Scenario.objects.get(name="Minimal Scenario")
+        # Default description from model is "no description"
+        self.assertIn(scenario.description.lower(), ["", "no description"])
+
+    def test_create_scenario_unicode_characters(self):
+        """Test that unicode characters are handled properly."""
+        self.client.force_login(self.user)
+
+        payload = {
+            "name": "Test Scenario 测试 🧪",
+            "description": "Description with émojis and spëcial çhars",
+            "scenario_date": "2024-01-01",
+            "scenario_type": "biodegradation",
+            "additional_information": [],
+        }
+
+        response = self.client.post(
+            f"/api/v1/package/{self.package.uuid}/scenario/",
+            data=json.dumps(payload),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, 200)
+        data = response.json()
+        self.assertIn("测试", data["name"])
+        self.assertIn("émojis", data["description"])

epapi/tests/v1/test_schema_generation.py

@@ -0,0 +1,114 @@
+"""
+Property-based tests for schema generation.
+
+Tests that verify schema generation works correctly for all models,
+regardless of their structure.
+"""
+
+import pytest
+from typing import Type
+from pydantic import BaseModel
+
+from envipy_additional_information import registry, EnviPyModel
+from epapi.utils.schema_transformers import build_rjsf_output
+
+
+class TestSchemaGeneration:
+    """Test that all models can generate valid RJSF schemas."""
+
+    @pytest.mark.parametrize("model_name,model_cls", list(registry.list_models().items()))
+    def test_all_models_generate_rjsf(self, model_name: str, model_cls: Type[BaseModel]):
+        """Every model in the registry should generate valid RJSF format."""
+        # Skip non-EnviPyModel classes (parsers, etc.)
+        if not issubclass(model_cls, EnviPyModel):
+            pytest.skip(f"{model_name} is not an EnviPyModel")
+
+        # Should not raise exception
+        result = build_rjsf_output(model_cls)
+
+        # Verify structure
+        assert isinstance(result, dict), f"{model_name}: Result should be a dict"
+        assert "schema" in result, f"{model_name}: Missing 'schema' key"
+        assert "uiSchema" in result, f"{model_name}: Missing 'uiSchema' key"
+        assert "formData" in result, f"{model_name}: Missing 'formData' key"
+        assert "groups" in result, f"{model_name}: Missing 'groups' key"
+
+        # Verify types
+        assert isinstance(result["schema"], dict), f"{model_name}: schema should be dict"
+        assert isinstance(result["uiSchema"], dict), f"{model_name}: uiSchema should be dict"
+        assert isinstance(result["formData"], dict), f"{model_name}: formData should be dict"
+        assert isinstance(result["groups"], list), f"{model_name}: groups should be list"
+
+        # Verify schema has properties
+        assert "properties" in result["schema"], f"{model_name}: schema should have 'properties'"
+        assert isinstance(result["schema"]["properties"], dict), (
+            f"{model_name}: properties should be dict"
+        )
+
+    @pytest.mark.parametrize("model_name,model_cls", list(registry.list_models().items()))
+    def test_ui_schema_matches_schema_fields(self, model_name: str, model_cls: Type[BaseModel]):
+        """uiSchema keys should match schema properties (or be nested for intervals)."""
+        if not issubclass(model_cls, EnviPyModel):
+            pytest.skip(f"{model_name} is not an EnviPyModel")
+
+        result = build_rjsf_output(model_cls)
+        schema_props = set(result["schema"]["properties"].keys())
+        ui_schema_keys = set(result["uiSchema"].keys())
+
+        # uiSchema should have entries for all top-level properties
+        # (intervals may have nested start/end, but the main field should be present)
+        assert ui_schema_keys.issubset(schema_props), (
+            f"{model_name}: uiSchema has keys not in schema: {ui_schema_keys - schema_props}"
+        )
+
+    @pytest.mark.parametrize("model_name,model_cls", list(registry.list_models().items()))
+    def test_groups_is_list_of_strings(self, model_name: str, model_cls: Type[BaseModel]):
+        """Groups should be a list of strings."""
+        if not issubclass(model_cls, EnviPyModel):
+            pytest.skip(f"{model_name} is not an EnviPyModel")
+
+        result = build_rjsf_output(model_cls)
+        groups = result["groups"]
+
+        assert isinstance(groups, list), f"{model_name}: groups should be list"
+        assert all(isinstance(g, str) for g in groups), (
+            f"{model_name}: all groups should be strings, got {groups}"
+        )
+        assert len(groups) > 0, f"{model_name}: should have at least one group"
+
+    @pytest.mark.parametrize("model_name,model_cls", list(registry.list_models().items()))
+    def test_form_data_matches_schema(self, model_name: str, model_cls: Type[BaseModel]):
+        """formData keys should match schema properties."""
+        if not issubclass(model_cls, EnviPyModel):
+            pytest.skip(f"{model_name} is not an EnviPyModel")
+
+        result = build_rjsf_output(model_cls)
+        schema_props = set(result["schema"]["properties"].keys())
+        form_data_keys = set(result["formData"].keys())
+
+        # formData should only contain keys that are in schema
+        assert form_data_keys.issubset(schema_props), (
+            f"{model_name}: formData has keys not in schema: {form_data_keys - schema_props}"
+        )
+
+
+class TestWidgetTypes:
+    """Test that widget types are valid."""
+
+    @pytest.mark.parametrize("model_name,model_cls", list(registry.list_models().items()))
+    def test_widget_types_are_valid(self, model_name: str, model_cls: Type[BaseModel]):
+        """All widget types in uiSchema should be valid WidgetType values."""
+        from envipy_additional_information.ui_config import WidgetType
+
+        if not issubclass(model_cls, EnviPyModel):
+            pytest.skip(f"{model_name} is not an EnviPyModel")
+
+        result = build_rjsf_output(model_cls)
+        valid_widgets = {wt.value for wt in WidgetType}
+
+        for field_name, ui_config in result["uiSchema"].items():
+            widget = ui_config.get("ui:widget")
+            if widget:
+                assert widget in valid_widgets, (
+                    f"{model_name}.{field_name}: Invalid widget '{widget}'. Valid: {valid_widgets}"
+                )

epapi/tests/v1/test_token_auth.py

@@ -0,0 +1,94 @@
+from datetime import timedelta
+
+from django.test import TestCase, tag
+from django.utils import timezone
+
+from epdb.logic import PackageManager, UserManager
+from epdb.models import APIToken
+
+
+@tag("api", "auth")
+class BearerTokenAuthTests(TestCase):
+    @classmethod
+    def setUpTestData(cls):
+        cls.user = UserManager.create_user(
+            "token-user",
+            "token-user@envipath.com",
+            "SuperSafe",
+            set_setting=False,
+            add_to_group=False,
+            is_active=True,
+        )
+
+        default_pkg = cls.user.default_package
+        cls.user.default_package = None
+        cls.user.save()
+        if default_pkg:
+            default_pkg.delete()
+
+        cls.unreviewed_package = PackageManager.create_package(
+            cls.user, "Token Auth Package", "Package for token auth tests"
+        )
+
+    def _auth_header(self, raw_token):
+        return {"HTTP_AUTHORIZATION": f"Bearer {raw_token}"}
+
+    def test_valid_token_allows_access(self):
+        _, raw_token = APIToken.create_token(self.user, name="Valid Token", expires_days=1)
+
+        response = self.client.get("/api/v1/compounds/", **self._auth_header(raw_token))
+
+        self.assertEqual(response.status_code, 200)
+
+    def test_expired_token_rejected(self):
+        token, raw_token = APIToken.create_token(self.user, name="Expired Token", expires_days=1)
+        token.expires_at = timezone.now() - timedelta(days=1)
+        token.save(update_fields=["expires_at"])
+
+        response = self.client.get("/api/v1/compounds/", **self._auth_header(raw_token))
+
+        self.assertEqual(response.status_code, 401)
+
+    def test_inactive_token_rejected(self):
+        token, raw_token = APIToken.create_token(self.user, name="Inactive Token", expires_days=1)
+        token.is_active = False
+        token.save(update_fields=["is_active"])
+
+        response = self.client.get("/api/v1/compounds/", **self._auth_header(raw_token))
+
+        self.assertEqual(response.status_code, 401)
+
+    def test_invalid_token_rejected(self):
+        response = self.client.get("/api/v1/compounds/", HTTP_AUTHORIZATION="Bearer invalid-token")
+
+        self.assertEqual(response.status_code, 401)
+
+    def test_no_token_rejected(self):
+        self.client.logout()
+        response = self.client.get("/api/v1/compounds/")
+
+        self.assertEqual(response.status_code, 401)
+
+    def test_bearer_populates_request_user_for_packages(self):
+        response = self.client.get("/api/v1/packages/")
+        self.assertEqual(response.status_code, 200)
+        payload = response.json()
+        uuids = {item["uuid"] for item in payload["items"]}
+        self.assertNotIn(str(self.unreviewed_package.uuid), uuids)
+
+        _, raw_token = APIToken.create_token(self.user, name="Package Token", expires_days=1)
+        response = self.client.get("/api/v1/packages/", **self._auth_header(raw_token))
+
+        self.assertEqual(response.status_code, 200)
+        payload = response.json()
+        uuids = {item["uuid"] for item in payload["items"]}
+        self.assertIn(str(self.unreviewed_package.uuid), uuids)
+
+    def test_session_auth_still_works_without_bearer(self):
+        self.client.force_login(self.user)
+        response = self.client.get("/api/v1/packages/")
+
+        self.assertEqual(response.status_code, 200)
+        payload = response.json()
+        uuids = {item["uuid"] for item in payload["items"]}
+        self.assertIn(str(self.unreviewed_package.uuid), uuids)

epapi/utils/schema_transformers.py

@@ -0,0 +1,181 @@
+"""
+Schema transformation utilities for converting Pydantic models to RJSF format.
+
+This module provides functions to extract UI configuration from Pydantic models
+and transform them into React JSON Schema Form (RJSF) compatible format.
+"""
+
+from typing import Type, Optional, Any
+
+import jsonref
+from pydantic import BaseModel
+
+from envipy_additional_information.ui_config import UIConfig
+from envipy_additional_information import registry
+
+
+def extract_groups(model_cls: Type[BaseModel]) -> list[str]:
+    """
+    Extract groups from registry-stored group information.
+
+    Args:
+        model_cls: The model class
+
+    Returns:
+        List of group names the model belongs to
+    """
+    return registry.get_groups(model_cls)
+
+
+def extract_ui_metadata(model_cls: Type[BaseModel]) -> dict[str, Any]:
+    """
+    Extract model-level UI metadata from UI class.
+
+    Returns metadata attributes that are NOT UIConfig instances.
+    Common metadata includes: unit, description, title.
+    """
+    metadata: dict[str, Any] = {}
+
+    if not hasattr(model_cls, "UI"):
+        return metadata
+
+    ui_class = getattr(model_cls, "UI")
+
+    # Iterate over all attributes in the UI class
+    for attr_name in dir(ui_class):
+        # Skip private attributes
+        if attr_name.startswith("_"):
+            continue
+
+        # Get the attribute value
+        try:
+            attr_value = getattr(ui_class, attr_name)
+        except AttributeError:
+            continue
+
+        # Skip callables but keep types/classes
+        if callable(attr_value) and not isinstance(attr_value, type):
+            continue
+
+        # Skip UIConfig instances (these are field-level configs, not metadata)
+        # This includes both UIConfig and IntervalConfig
+        if isinstance(attr_value, UIConfig):
+            continue
+
+        metadata[attr_name] = attr_value
+
+    return metadata
+
+
+def extract_ui_config_from_model(model_cls: Type[BaseModel]) -> dict[str, Any]:
+    """
+    Extract UI configuration from model's UI class.
+
+    Returns a dictionary mapping field names to their UI schema configurations.
+    Trusts the config classes to handle their own transformation logic.
+    """
+    ui_configs: dict[str, Any] = {}
+
+    if not hasattr(model_cls, "UI"):
+        return ui_configs
+
+    ui_class = getattr(model_cls, "UI")
+    schema = model_cls.model_json_schema()
+    field_names = schema.get("properties", {}).keys()
+
+    # Extract config for each field
+    for field_name in field_names:
+        # Skip if UI config doesn't exist for this field (field may be hidden from UI)
+        if not hasattr(ui_class, field_name):
+            continue
+
+        ui_config = getattr(ui_class, field_name)
+
+        if isinstance(ui_config, UIConfig):
+            ui_configs[field_name] = ui_config.to_ui_schema_field()
+
+    return ui_configs
+
+
+def build_ui_schema(model_cls: Type[BaseModel]) -> dict:
+    """Generate RJSF uiSchema from model's UI class."""
+    ui_schema = {}
+
+    # Extract field-level UI configs
+    field_configs = extract_ui_config_from_model(model_cls)
+
+    for field_name, config in field_configs.items():
+        ui_schema[field_name] = config
+
+    return ui_schema
+
+
+def build_schema(model_cls: Type[BaseModel]) -> dict[str, Any]:
+    """
+    Build JSON schema from Pydantic model, applying UI metadata.
+
+    Dereferences all $ref pointers to produce fully inlined schema.
+    This ensures the frontend receives schemas with enum values and nested
+    properties fully resolved, without needing client-side ref resolution.
+
+    Extracts model-level metadata from UI class (title, unit, etc.) and applies
+    it to the generated schema. This ensures UI metadata is the single source of truth.
+    """
+    schema = model_cls.model_json_schema()
+
+    # Dereference $ref pointers (inlines $defs) using jsonref
+    # This ensures the frontend receives schemas with enum values and nested
+    # properties fully resolved, currently necessary for client-side rendering.
+    # FIXME: This is a hack to get the schema to work with alpine schema-form.js replace once we migrate to client-side framework.
+    schema = jsonref.replace_refs(schema, proxies=False)
+
+    # Remove $defs section as all refs are now inlined
+    if "$defs" in schema:
+        del schema["$defs"]
+
+    # Extract and apply UI metadata (title, unit, description, etc.)
+    ui_metadata = extract_ui_metadata(model_cls)
+
+    # Apply all metadata consistently as custom properties with x- prefix
+    # This ensures consistency and avoids conflicts with standard JSON Schema properties
+    for key, value in ui_metadata.items():
+        if value is not None:
+            schema[f"x-{key}"] = value
+
+    # Set standard title property from UI metadata for JSON Schema compliance
+    if "title" in ui_metadata:
+        schema["title"] = ui_metadata["title"]
+    elif "label" in ui_metadata:
+        schema["title"] = ui_metadata["label"]
+
+    return schema
+
+
+def build_rjsf_output(model_cls: Type[BaseModel], initial_data: Optional[dict] = None) -> dict:
+    """
+    Main function that returns complete RJSF format.
+
+    Trusts the config classes to handle their own transformation logic.
+    No special-case handling - if a config knows how to transform itself, it will.
+
+    Returns:
+        dict with keys: schema, uiSchema, formData, groups
+    """
+    # Build schema with UI metadata applied
+    schema = build_schema(model_cls)
+
+    # Build UI schema - config classes handle their own transformation
+    ui_schema = build_ui_schema(model_cls)
+
+    # Extract groups from marker interfaces
+    groups = extract_groups(model_cls)
+
+    # Use provided initial_data or empty dict
+    form_data = initial_data if initial_data is not None else {}
+
+    return {
+        "schema": schema,
+        "uiSchema": ui_schema,
+        "formData": form_data,
+        "groups": groups,
+    }

epapi/utils/validation_errors.py

@@ -0,0 +1,82 @@
+"""Shared utilities for handling Pydantic validation errors."""
+
+import json
+from pydantic import ValidationError
+from pydantic_core import ErrorDetails
+from ninja.errors import HttpError
+
+
+def format_validation_error(error: ErrorDetails) -> str:
+    """Format a Pydantic validation error into a user-friendly message.
+
+    Args:
+        error: A Pydantic error details dictionary containing 'msg', 'type', 'ctx', etc.
+
+    Returns:
+        A user-friendly error message string.
+    """
+    msg = error.get("msg") or "Invalid value"
+    error_type = error.get("type") or ""
+
+    # Handle common validation types with friendly messages
+    if error_type == "enum":
+        ctx = error.get("ctx", {})
+        expected = ctx.get("expected", "") if ctx else ""
+        return f"Please select a valid option{': ' + expected if expected else ''}"
+    elif error_type == "literal_error":
+        # Literal errors (like Literal["active", "inactive"])
+        return msg.replace("Input should be ", "Please enter ")
+    elif error_type == "missing":
+        return "This field is required"
+    elif error_type == "string_type":
+        return "Please enter a valid string"
+    elif error_type == "int_type":
+        return "Please enter a valid int"
+    elif error_type == "int_parsing":
+        return "Please enter a valid int"
+    elif error_type == "float_type":
+        return "Please enter a valid float"
+    elif error_type == "float_parsing":
+        return "Please enter a valid float"
+    elif error_type == "value_error":
+        # Strip "Value error, " prefix from custom validator messages
+        return msg.replace("Value error, ", "")
+    else:
+        # Default: use the message from Pydantic but clean it up
+        return msg.replace("Input should be ", "Please enter ").replace("Value error, ", "")
+
+
+def handle_validation_error(e: ValidationError) -> None:
+    """Convert a Pydantic ValidationError into a structured HttpError.
+
+    This function transforms Pydantic validation errors into a JSON structure
+    that the frontend expects for displaying field-level errors.
+
+    Args:
+        e: The Pydantic ValidationError to handle.
+
+    Raises:
+        HttpError: Always raises a 400 error with structured JSON containing
+                   type, field_errors, and message fields.
+    """
+    # Transform Pydantic validation errors into user-friendly format
+    field_errors: dict[str, list[str]] = {}
+    for error in e.errors():
+        # Get the field name from location tuple
+        loc = error.get("loc", ())
+        field = str(loc[-1]) if loc else "root"
+
+        # Format the error message
+        friendly_msg = format_validation_error(error)
+
+        if field not in field_errors:
+            field_errors[field] = []
+        field_errors[field].append(friendly_msg)
+
+    # Return structured error for frontend parsing
+    error_response = {
+        "type": "validation_error",
+        "field_errors": field_errors,
+        "message": "Please correct the errors below",
+    }
+    raise HttpError(400, json.dumps(error_response))

epapi/v1/auth.py

@@ -1,8 +1,34 @@
+import hashlib
+
 from ninja.security import HttpBearer
 from ninja.errors import HttpError
 
+from epdb.models import APIToken
+
 
 class BearerTokenAuth(HttpBearer):
     def authenticate(self, request, token):
-        # FIXME: placeholder; implement it in O(1) time
-        raise HttpError(401, "Invalid or expired token")
+        if token is None:
+            return None
+
+        hashed_token = hashlib.sha256(token.encode()).hexdigest()
+        user = APIToken.authenticate(hashed_token, hashed=True)
+        if not user:
+            raise HttpError(401, "Invalid or expired token")
+
+        request.user = user
+        return user
+
+
+class OptionalBearerTokenAuth:
+    """Bearer auth that allows unauthenticated access.
+
+    Validates the Bearer token if present (401 on invalid token),
+    otherwise lets the request through for anonymous/session access.
+    """
+
+    def __init__(self):
+        self._bearer = BearerTokenAuth()
+
+    def __call__(self, request):
+        return self._bearer(request) or request.user

epapi/v1/dal.py

@@ -1,12 +1,12 @@
 from django.db.models import Model
 from epdb.logic import PackageManager
-from epdb.models import CompoundStructure, User, Package, Compound
+from epdb.models import CompoundStructure, User, Package, Compound, Scenario
 from uuid import UUID
 
 from .errors import EPAPINotFoundError, EPAPIPermissionDeniedError
 
 
-def get_compound_or_error(user, compound_uuid: UUID):
+def get_compound_for_read(user, compound_uuid: UUID):
     """
     Get compound by UUID with permission check.
     """
@@ -23,7 +23,7 @@ def get_compound_or_error(user, compound_uuid: UUID):
     return compound
 
 
-def get_package_or_error(user, package_uuid: UUID):
+def get_package_for_read(user, package_uuid: UUID):
     """
     Get package by UUID with permission check.
     """
@@ -41,14 +41,58 @@ def get_package_or_error(user, package_uuid: UUID):
     return package
 
 
-def get_user_packages_qs(user: User | None):
+def get_package_for_write(user, package_uuid: UUID):
+    """
+    Get package by UUID with permission check.
+    """
+
+    # FIXME: update package manager with custom exceptions to avoid manual checks here
+    try:
+        package = Package.objects.get(uuid=package_uuid)
+    except Package.DoesNotExist:
+        raise EPAPINotFoundError(f"Package with UUID {package_uuid} not found")
+
+    # FIXME: optimize package manager to exclusively work with UUIDs
+    if not user or user.is_anonymous or not PackageManager.writable(user, package):
+        raise EPAPIPermissionDeniedError("Insufficient permissions to access this package.")
+
+    return package
+
+
+def get_scenario_for_read(user, scenario_uuid: UUID):
+    """Get scenario by UUID with read permission check."""
+    try:
+        scenario = Scenario.objects.select_related("package").get(uuid=scenario_uuid)
+    except Scenario.DoesNotExist:
+        raise EPAPINotFoundError(f"Scenario with UUID {scenario_uuid} not found")
+
+    if not user or user.is_anonymous or not PackageManager.readable(user, scenario.package):
+        raise EPAPIPermissionDeniedError("Insufficient permissions to access this scenario.")
+
+    return scenario
+
+
+def get_scenario_for_write(user, scenario_uuid: UUID):
+    """Get scenario by UUID with write permission check."""
+    try:
+        scenario = Scenario.objects.select_related("package").get(uuid=scenario_uuid)
+    except Scenario.DoesNotExist:
+        raise EPAPINotFoundError(f"Scenario with UUID {scenario_uuid} not found")
+
+    if not user or user.is_anonymous or not PackageManager.writable(user, scenario.package):
+        raise EPAPIPermissionDeniedError("Insufficient permissions to modify this scenario.")
+
+    return scenario
+
+
+def get_user_packages_for_read(user: User | None):
     """Get all packages readable by the user."""
     if not user or user.is_anonymous:
         return PackageManager.get_reviewed_packages()
     return PackageManager.get_all_readable_packages(user, include_reviewed=True)
 
 
-def get_user_entities_qs(model_class: Model, user: User | None):
+def get_user_entities_for_read(model_class: Model, user: User | None):
     """Build queryset for reviewed package entities."""
 
     if not user or user.is_anonymous:
@@ -60,16 +104,14 @@ def get_user_entities_qs(model_class: Model, user: User | None):
     return qs
 
 
-def get_package_scoped_entities_qs(
-    model_class: Model, package_uuid: UUID, user: User | None = None
-):
+def get_package_entities_for_read(model_class: Model, package_uuid: UUID, user: User | None = None):
     """Build queryset for specific package entities."""
-    package = get_package_or_error(user, package_uuid)
+    package = get_package_for_read(user, package_uuid)
     qs = model_class.objects.filter(package=package).select_related("package")
     return qs
 
 
-def get_user_structures_qs(user: User | None):
+def get_user_structure_for_read(user: User | None):
     """Build queryset for structures accessible to the user (via compound->package)."""
 
     if not user or user.is_anonymous:
@@ -83,13 +125,13 @@ def get_user_structures_qs(user: User | None):
     return qs
 
 
-def get_package_compound_scoped_structure_qs(
+def get_package_compound_structure_for_read(
     package_uuid: UUID, compound_uuid: UUID, user: User | None = None
 ):
     """Build queryset for specific package compound structures."""
 
-    get_package_or_error(user, package_uuid)
-    compound = get_compound_or_error(user, compound_uuid)
+    get_package_for_read(user, package_uuid)
+    compound = get_compound_for_read(user, compound_uuid)
 
     qs = CompoundStructure.objects.filter(compound=compound).select_related("compound__package")
     return qs

epapi/v1/endpoints/additional_information.py

@@ -0,0 +1,174 @@
+from ninja import Router, Body
+from ninja.errors import HttpError
+from uuid import UUID
+from pydantic import ValidationError
+from typing import Dict, Any
+import logging
+
+from envipy_additional_information import registry
+from envipy_additional_information.groups import GroupEnum
+from epapi.utils.schema_transformers import build_rjsf_output
+from epapi.utils.validation_errors import handle_validation_error
+from epdb.models import AdditionalInformation
+from ..dal import get_scenario_for_read, get_scenario_for_write
+
+logger = logging.getLogger(__name__)
+
+router = Router(tags=["Additional Information"])
+
+
+@router.get("/information/schema/")
+def list_all_schemas(request):
+    """Return all schemas in RJSF format with lowercase class names as keys."""
+    result = {}
+    for name, cls in registry.list_models().items():
+        try:
+            result[name] = build_rjsf_output(cls)
+        except Exception as e:
+            logger.warning(f"Failed to generate schema for {name}: {e}")
+            continue
+    return result
+
+
+@router.get("/information/schema/{model_name}/")
+def get_model_schema(request, model_name: str):
+    """Return RJSF schema for specific model."""
+    cls = registry.get_model(model_name.lower())
+    if not cls:
+        raise HttpError(404, f"Unknown model: {model_name}")
+    return build_rjsf_output(cls)
+
+
+@router.get("/scenario/{uuid:scenario_uuid}/information/")
+def list_scenario_info(request, scenario_uuid: UUID):
+    """List all additional information for a scenario"""
+    scenario = get_scenario_for_read(request.user, scenario_uuid)
+
+    result = []
+
+    for ai in AdditionalInformation.objects.filter(scenario=scenario):
+        result.append(
+            {
+                "type": ai.get().__class__.__name__,
+                "uuid": getattr(ai, "uuid", None),
+                "data": ai.data,
+                "attach_object": ai.content_object.simple_json() if ai.content_object else None,
+            }
+        )
+    return result
+
+
+@router.post("/scenario/{uuid:scenario_uuid}/information/{model_name}/")
+def add_scenario_info(
+    request, scenario_uuid: UUID, model_name: str, payload: Dict[str, Any] = Body(...)
+):
+    """Add new additional information to scenario"""
+    cls = registry.get_model(model_name.lower())
+    if not cls:
+        raise HttpError(404, f"Unknown model: {model_name}")
+
+    try:
+        instance = cls(**payload)  # Pydantic validates
+    except ValidationError as e:
+        handle_validation_error(e)
+
+    scenario = get_scenario_for_write(request.user, scenario_uuid)
+
+    # Model method now returns the UUID
+    created_uuid = scenario.add_additional_information(instance)
+
+    return {"status": "created", "uuid": created_uuid}
+
+
+@router.patch("/scenario/{uuid:scenario_uuid}/information/item/{uuid:ai_uuid}/")
+def update_scenario_info(
+    request, scenario_uuid: UUID, ai_uuid: UUID, payload: Dict[str, Any] = Body(...)
+):
+    """Update existing additional information for a scenario"""
+    scenario = get_scenario_for_write(request.user, scenario_uuid)
+    ai_uuid_str = str(ai_uuid)
+
+    ai = AdditionalInformation.objects.filter(uuid=ai_uuid_str, scenario=scenario)
+
+    if not ai.exists():
+        raise HttpError(404, f"Additional information with UUID {ai_uuid} not found")
+
+    ai = ai.first()
+
+    # Get the model class for validation
+    cls = registry.get_model(ai.type.lower())
+    if not cls:
+        raise HttpError(500, f"Unknown model type in data: {ai.type}")
+
+    # Validate the payload against the model
+    try:
+        instance = cls(**payload)
+    except ValidationError as e:
+        handle_validation_error(e)
+
+    # Use model method for update
+    try:
+        scenario.update_additional_information(ai_uuid_str, instance)
+    except ValueError as e:
+        raise HttpError(404, str(e))
+
+    return {"status": "updated", "uuid": ai_uuid_str}
+
+
+@router.delete("/scenario/{uuid:scenario_uuid}/information/item/{uuid:ai_uuid}/")
+def delete_scenario_info(request, scenario_uuid: UUID, ai_uuid: UUID):
+    """Delete additional information from scenario"""
+    scenario = get_scenario_for_write(request.user, scenario_uuid)
+
+    try:
+        scenario.remove_additional_information(str(ai_uuid))
+    except ValueError as e:
+        raise HttpError(404, str(e))
+
+    return {"status": "deleted"}
+
+
+@router.get("/information/groups/")
+def list_groups(request):
+    """Return list of available group names."""
+    return {"groups": GroupEnum.values()}
+
+
+@router.get("/information/groups/{group_name}/")
+def get_group_models(request, group_name: str):
+    """
+    Return models for a specific group organized by subcategory.
+
+    Args:
+        group_name: One of "sludge", "soil", or "sediment" (string)
+
+    Returns:
+        Dictionary with subcategories (exp, spike, comp, misc, or group name)
+        as keys and lists of model info as values
+    """
+    # Convert string to enum (raises ValueError if invalid)
+    try:
+        group_enum = GroupEnum(group_name)
+    except ValueError:
+        valid = ", ".join(GroupEnum.values())
+        raise HttpError(400, f"Invalid group '{group_name}'. Valid: {valid}")
+
+    try:
+        group_data = registry.collect_group(group_enum)
+    except (ValueError, TypeError) as e:
+        raise HttpError(400, str(e))
+
+    result = {}
+    for subcategory, models in group_data.items():
+        result[subcategory] = [
+            {
+                "name": cls.__name__.lower(),
+                "class": cls.__name__,
+                "title": getattr(cls.UI, "title", cls.__name__)
+                if hasattr(cls, "UI")
+                else cls.__name__,
+            }
+            for cls in models
+        ]
+
+    return result

epapi/v1/endpoints/compounds.py

@@ -6,7 +6,7 @@ from uuid import UUID
 from epdb.models import Compound
 from ..pagination import EnhancedPageNumberPagination
 from ..schemas import CompoundOutSchema, ReviewStatusFilter
-from ..dal import get_user_entities_qs, get_package_scoped_entities_qs
+from ..dal import get_user_entities_for_read, get_package_entities_for_read
 
 router = Router()
 
@@ -21,7 +21,7 @@ def list_all_compounds(request):
     """
     List all compounds from reviewed packages.
     """
-    return get_user_entities_qs(Compound, request.user).order_by("name").all()
+    return get_user_entities_for_read(Compound, request.user).order_by("name").all()
 
 
 @router.get(
@@ -38,4 +38,4 @@ def list_package_compounds(request, package_uuid: UUID):
     List all compounds for a specific package.
     """
     user = request.user
-    return get_package_scoped_entities_qs(Compound, package_uuid, user).order_by("name").all()
+    return get_package_entities_for_read(Compound, package_uuid, user).order_by("name").all()

epapi/v1/endpoints/models.py

@@ -6,7 +6,7 @@ from uuid import UUID
 from epdb.models import EPModel
 from ..pagination import EnhancedPageNumberPagination
 from ..schemas import ModelOutSchema, ReviewStatusFilter
-from ..dal import get_user_entities_qs, get_package_scoped_entities_qs
+from ..dal import get_user_entities_for_read, get_package_entities_for_read
 
 router = Router()
 
@@ -21,7 +21,7 @@ def list_all_models(request):
     """
     List all models from reviewed packages.
     """
-    return get_user_entities_qs(EPModel, request.user).order_by("name").all()
+    return get_user_entities_for_read(EPModel, request.user).order_by("name").all()
 
 
 @router.get(
@@ -38,4 +38,4 @@ def list_package_models(request, package_uuid: UUID):
     List all models for a specific package.
     """
     user = request.user
-    return get_package_scoped_entities_qs(EPModel, package_uuid, user).order_by("name").all()
+    return get_package_entities_for_read(EPModel, package_uuid, user).order_by("name").all()

epapi/v1/endpoints/packages.py

@@ -3,7 +3,8 @@ from ninja import Router
 from ninja_extra.pagination import paginate
 import logging
 
-from ..dal import get_user_packages_qs
+from ..auth import OptionalBearerTokenAuth
+from ..dal import get_user_packages_for_read
 from ..pagination import EnhancedPageNumberPagination
 from ..schemas import PackageOutSchema, SelfReviewStatusFilter
 
@@ -11,7 +12,11 @@ router = Router()
 logger = logging.getLogger(__name__)
 
 
-@router.get("/packages/", response=EnhancedPageNumberPagination.Output[PackageOutSchema], auth=None)
+@router.get(
+    "/packages/",
+    response=EnhancedPageNumberPagination.Output[PackageOutSchema],
+    auth=OptionalBearerTokenAuth(),
+)
 @paginate(
     EnhancedPageNumberPagination,
     page_size=s.API_PAGINATION_DEFAULT_PAGE_SIZE,
@@ -23,5 +28,5 @@ def list_all_packages(request):
 
     """
     user = request.user
-    qs = get_user_packages_qs(user)
+    qs = get_user_packages_for_read(user)
     return qs.order_by("name").all()

epapi/v1/endpoints/pathways.py

@@ -6,7 +6,7 @@ from uuid import UUID
 from epdb.models import Pathway
 from ..pagination import EnhancedPageNumberPagination
 from ..schemas import PathwayOutSchema, ReviewStatusFilter
-from ..dal import get_user_entities_qs, get_package_scoped_entities_qs
+from ..dal import get_user_entities_for_read, get_package_entities_for_read
 
 router = Router()
 
@@ -22,7 +22,7 @@ def list_all_pathways(request):
     List all pathways from reviewed packages.
     """
     user = request.user
-    return get_user_entities_qs(Pathway, user).order_by("name").all()
+    return get_user_entities_for_read(Pathway, user).order_by("name").all()
 
 
 @router.get(
@@ -39,4 +39,4 @@ def list_package_pathways(request, package_uuid: UUID):
     List all pathways for a specific package.
     """
     user = request.user
-    return get_package_scoped_entities_qs(Pathway, package_uuid, user).order_by("name").all()
+    return get_package_entities_for_read(Pathway, package_uuid, user).order_by("name").all()

epapi/v1/endpoints/reactions.py

@@ -6,7 +6,7 @@ from uuid import UUID
 from epdb.models import Reaction
 from ..pagination import EnhancedPageNumberPagination
 from ..schemas import ReactionOutSchema, ReviewStatusFilter
-from ..dal import get_user_entities_qs, get_package_scoped_entities_qs
+from ..dal import get_user_entities_for_read, get_package_entities_for_read
 
 router = Router()
 
@@ -22,7 +22,7 @@ def list_all_reactions(request):
     List all reactions from reviewed packages.
     """
     user = request.user
-    return get_user_entities_qs(Reaction, user).order_by("name").all()
+    return get_user_entities_for_read(Reaction, user).order_by("name").all()
 
 
 @router.get(
@@ -39,4 +39,4 @@ def list_package_reactions(request, package_uuid: UUID):
     List all reactions for a specific package.
     """
     user = request.user
-    return get_package_scoped_entities_qs(Reaction, package_uuid, user).order_by("name").all()
+    return get_package_entities_for_read(Reaction, package_uuid, user).order_by("name").all()

epapi/v1/endpoints/rules.py

@@ -6,7 +6,7 @@ from uuid import UUID
 from epdb.models import Rule
 from ..pagination import EnhancedPageNumberPagination
 from ..schemas import ReviewStatusFilter, RuleOutSchema
-from ..dal import get_user_entities_qs, get_package_scoped_entities_qs
+from ..dal import get_user_entities_for_read, get_package_entities_for_read
 
 router = Router()
 
@@ -22,7 +22,7 @@ def list_all_rules(request):
     List all rules from reviewed packages.
     """
     user = request.user
-    return get_user_entities_qs(Rule, user).order_by("name").all()
+    return get_user_entities_for_read(Rule, user).order_by("name").all()
 
 
 @router.get(
@@ -39,4 +39,4 @@ def list_package_rules(request, package_uuid: UUID):
     List all rules for a specific package.
     """
     user = request.user
-    return get_package_scoped_entities_qs(Rule, package_uuid, user).order_by("name").all()
+    return get_package_entities_for_read(Rule, package_uuid, user).order_by("name").all()

epapi/v1/endpoints/scenarios.py

@@ -1,12 +1,25 @@
 from django.conf import settings as s
-from ninja import Router
+from django.db import IntegrityError, OperationalError, DatabaseError
+from ninja import Router, Body
+from ninja.errors import HttpError
 from ninja_extra.pagination import paginate
 from uuid import UUID
+from pydantic import ValidationError
+import logging
+import json
 
 from epdb.models import Scenario
+from epdb.views import _anonymous_or_real
 from ..pagination import EnhancedPageNumberPagination
-from ..schemas import ReviewStatusFilter, ScenarioOutSchema
-from ..dal import get_user_entities_qs, get_package_scoped_entities_qs
+from ..schemas import (
+    ReviewStatusFilter,
+    ScenarioOutSchema,
+    ScenarioCreateSchema,
+)
+from ..dal import get_user_entities_for_read, get_package_entities_for_read, get_package_for_write
+from envipy_additional_information import registry
+
+logger = logging.getLogger(__name__)
 
 router = Router()
 
@@ -19,7 +32,8 @@ router = Router()
 )
 def list_all_scenarios(request):
     user = request.user
-    return get_user_entities_qs(Scenario, user).order_by("name").all()
+    items = get_user_entities_for_read(Scenario, user)
+    return items.order_by("name").all()
 
 
 @router.get(
@@ -33,4 +47,83 @@ def list_all_scenarios(request):
 )
 def list_package_scenarios(request, package_uuid: UUID):
     user = request.user
-    return get_package_scoped_entities_qs(Scenario, package_uuid, user).order_by("name").all()
+    items = get_package_entities_for_read(Scenario, package_uuid, user)
+    return items.order_by("name").all()
+
+
+@router.post("/package/{uuid:package_uuid}/scenario/", response=ScenarioOutSchema)
+def create_scenario(request, package_uuid: UUID, payload: ScenarioCreateSchema = Body(...)):
+    """Create a new scenario with optional additional information."""
+    user = _anonymous_or_real(request)
+
+    try:
+        current_package = get_package_for_write(user, package_uuid)
+    except ValueError as e:
+        error_msg = str(e)
+        if "does not exist" in error_msg:
+            raise HttpError(404, f"Package not found: {package_uuid}")
+        elif "Insufficient permissions" in error_msg:
+            raise HttpError(403, "You do not have permission to access this package")
+        else:
+            logger.error(f"Unexpected ValueError from get_package_by_id: {error_msg}")
+            raise HttpError(400, "Invalid package request")
+
+    # Build additional information models from payload
+    additional_information_models = []
+    validation_errors = []
+
+    for ai_item in payload.additional_information:
+        # Get model class from registry
+        model_cls = registry.get_model(ai_item.type.lower())
+        if not model_cls:
+            validation_errors.append(f"Unknown additional information type: {ai_item.type}")
+            continue
+
+        try:
+            # Validate and create model instance
+            instance = model_cls(**ai_item.data)
+            additional_information_models.append(instance)
+        except ValidationError as e:
+            # Collect validation errors to return to user
+            error_messages = [err.get("msg", "Validation error") for err in e.errors()]
+            validation_errors.append(f"{ai_item.type}: {', '.join(error_messages)}")
+        except (TypeError, AttributeError, KeyError) as e:
+            logger.warning(f"Failed to instantiate {ai_item.type} model: {str(e)}")
+            validation_errors.append(f"{ai_item.type}: Invalid data structure - {str(e)}")
+        except Exception as e:
+            logger.error(f"Unexpected error instantiating {ai_item.type}: {str(e)}")
+            validation_errors.append(f"{ai_item.type}: Failed to process - please check your data")
+
+    # If there are validation errors, return them
+    if validation_errors:
+        raise HttpError(
+            400,
+            json.dumps(
+                {
+                    "error": "Validation errors in additional information",
+                    "details": validation_errors,
+                }
+            ),
+        )
+
+    # Create scenario using the existing Scenario.create method
+    try:
+        new_scenario = Scenario.create(
+            package=current_package,
+            name=payload.name,
+            description=payload.description,
+            scenario_date=payload.scenario_date,
+            scenario_type=payload.scenario_type,
+            additional_information=additional_information_models,
+        )
+    except IntegrityError as e:
+        logger.error(f"Database integrity error creating scenario: {str(e)}")
+        raise HttpError(400, "Scenario creation failed - data constraint violation")
+    except OperationalError as e:
+        logger.error(f"Database operational error creating scenario: {str(e)}")
+        raise HttpError(503, "Database temporarily unavailable - please try again")
+    except (DatabaseError, AttributeError) as e:
+        logger.error(f"Error creating scenario: {str(e)}")
+        raise HttpError(500, "Failed to create scenario due to database error")
+
+    return new_scenario

epapi/v1/endpoints/settings.py

@@ -0,0 +1,23 @@
+from django.conf import settings as s
+from ninja import Router
+from ninja_extra.pagination import paginate
+
+from epdb.logic import SettingManager
+
+from ..pagination import EnhancedPageNumberPagination
+from ..schemas import SettingOutSchema
+
+router = Router()
+
+
+@router.get("/settings/", response=EnhancedPageNumberPagination.Output[SettingOutSchema])
+@paginate(
+    EnhancedPageNumberPagination,
+    page_size=s.API_PAGINATION_DEFAULT_PAGE_SIZE,
+)
+def list_all_pathways(request):
+    """
+    List all pathways from reviewed packages.
+    """
+    user = request.user
+    return SettingManager.get_all_settings(user)

epapi/v1/endpoints/structure.py

@@ -6,8 +6,8 @@ from uuid import UUID
 from ..pagination import EnhancedPageNumberPagination
 from ..schemas import CompoundStructureOutSchema, StructureReviewStatusFilter
 from ..dal import (
-    get_user_structures_qs,
-    get_package_compound_scoped_structure_qs,
+    get_user_structure_for_read,
+    get_package_compound_structure_for_read,
 )
 
 router = Router()
@@ -26,7 +26,7 @@ def list_all_structures(request):
     List all structures from all packages.
     """
     user = request.user
-    return get_user_structures_qs(user).order_by("name").all()
+    return get_user_structure_for_read(user).order_by("name").all()
 
 
 @router.get(
@@ -44,7 +44,7 @@ def list_package_structures(request, package_uuid: UUID, compound_uuid: UUID):
     """
     user = request.user
     return (
-        get_package_compound_scoped_structure_qs(package_uuid, compound_uuid, user)
+        get_package_compound_structure_for_read(package_uuid, compound_uuid, user)
         .order_by("name")
         .all()
     )

epapi/v1/router.py

@@ -1,7 +1,19 @@
 from ninja import Router
 from ninja.security import SessionAuth
+
 from .auth import BearerTokenAuth
-from .endpoints import packages, scenarios, compounds, rules, reactions, pathways, models, structure
+from .endpoints import (
+    packages,
+    scenarios,
+    compounds,
+    rules,
+    reactions,
+    pathways,
+    models,
+    structure,
+    additional_information,
+    settings,
+)
 
 # Main router with authentication
 router = Router(
@@ -20,3 +32,5 @@ router.add_router("", reactions.router)
 router.add_router("", pathways.router)
 router.add_router("", models.router)
 router.add_router("", structure.router)
+router.add_router("", additional_information.router)
+router.add_router("", settings.router)

epapi/v1/schemas.py

@@ -1,5 +1,5 @@
 from ninja import FilterSchema, FilterLookup, Schema
-from typing import Annotated, Optional
+from typing import Annotated, Optional, List, Dict, Any
 from uuid import UUID
 
 
@@ -51,6 +51,23 @@ class ScenarioOutSchema(PackageEntityOutSchema):
     pass
 
 
+class AdditionalInformationItemSchema(Schema):
+    """Schema for additional information item in scenario creation."""
+
+    type: str
+    data: Dict[str, Any]
+
+
+class ScenarioCreateSchema(Schema):
+    """Schema for creating a new scenario."""
+
+    name: str
+    description: str = ""
+    scenario_date: str = "No date"
+    scenario_type: str = "Not specified"
+    additional_information: List[AdditionalInformationItemSchema] = []
+
+
 class CompoundOutSchema(PackageEntityOutSchema):
     pass
 
@@ -102,3 +119,10 @@ class PackageOutSchema(Schema):
     @staticmethod
     def resolve_review_status(obj):
         return "reviewed" if obj.reviewed else "unreviewed"
+
+
+class SettingOutSchema(Schema):
+    uuid: UUID
+    url: str = ""
+    name: str
+    description: str

epdb/admin.py

@@ -2,6 +2,7 @@ from django.conf import settings as s
 from django.contrib import admin
 
 from .models import (
+    AdditionalInformation,
     Compound,
     CompoundStructure,
     Edge,
@@ -16,6 +17,7 @@ from .models import (
     Node,
     ParallelRule,
     Pathway,
+    PropertyPluginModel,
     Reaction,
     Scenario,
     Setting,
@@ -27,8 +29,20 @@ from .models import (
 Package = s.GET_PACKAGE_MODEL()
 
 
+class AdditionalInformationAdmin(admin.ModelAdmin):
+    pass
+
+
 class UserAdmin(admin.ModelAdmin):
-    list_display = ["username", "email", "is_active"]
+    list_display = [
+        "username",
+        "email",
+        "is_active",
+        "is_staff",
+        "is_superuser",
+        "last_login",
+        "date_joined",
+    ]
 
 
 class UserPackagePermissionAdmin(admin.ModelAdmin):
@@ -48,7 +62,7 @@ class JobLogAdmin(admin.ModelAdmin):
 
 
 class EPAdmin(admin.ModelAdmin):
-    search_fields = ["name", "description"]
+    search_fields = ["name", "description", "url", "uuid"]
     list_display = ["name", "url", "created"]
     ordering = ["-created"]
 
@@ -65,6 +79,10 @@ class EnviFormerAdmin(EPAdmin):
     pass
 
 
+class PropertyPluginModelAdmin(admin.ModelAdmin):
+    pass
+
+
 class LicenseAdmin(admin.ModelAdmin):
     list_display = ["cc_string", "link", "image_link"]
 
@@ -117,6 +135,7 @@ class ExternalIdentifierAdmin(admin.ModelAdmin):
     pass
 
 
+admin.site.register(AdditionalInformation, AdditionalInformationAdmin)
 admin.site.register(User, UserAdmin)
 admin.site.register(UserPackagePermission, UserPackagePermissionAdmin)
 admin.site.register(Group, GroupAdmin)
@@ -125,6 +144,7 @@ admin.site.register(JobLog, JobLogAdmin)
 admin.site.register(Package, PackageAdmin)
 admin.site.register(MLRelativeReasoning, MLRelativeReasoningAdmin)
 admin.site.register(EnviFormer, EnviFormerAdmin)
+admin.site.register(PropertyPluginModel, PropertyPluginModelAdmin)
 admin.site.register(License, LicenseAdmin)
 admin.site.register(Compound, CompoundAdmin)
 admin.site.register(CompoundStructure, CompoundStructureAdmin)

epdb/api.py

@@ -2,20 +2,12 @@ from typing import List
 
 from django.contrib.auth import get_user_model
 from ninja import Router, Schema, Field
-from ninja.errors import HttpError
 from ninja.pagination import paginate
-from ninja.security import HttpBearer
+
+from epapi.v1.auth import BearerTokenAuth
 
 from .logic import PackageManager
-from .models import User, Compound, APIToken
-
-
-class BearerTokenAuth(HttpBearer):
-    def authenticate(self, request, token):
-        for token_obj in APIToken.objects.select_related("user").all():
-            if token_obj.check_token(token) and token_obj.is_valid():
-                return token_obj.user
-        raise HttpError(401, "Invalid or expired token")
+from .models import User, Compound
 
 
 def _anonymous_or_real(request):

epdb/apps.py

@@ -15,3 +15,9 @@ class EPDBConfig(AppConfig):
 
         model_name = getattr(settings, "EPDB_PACKAGE_MODEL", "epdb.Package")
         logger.info(f"Using Package model: {model_name}")
+
+        if settings.PLUGINS_ENABLED:
+            from bridge.contracts import Property
+            from utilities.plugin import discover_plugins
+
+            settings.PROPERTY_PLUGINS.update(**discover_plugins(_cls=Property))

epdb/legacy_api.py

@@ -3,34 +3,47 @@ from typing import Any, Dict, List, Optional
 import nh3
 from django.conf import settings as s
 from django.contrib.auth import get_user_model
-from django.http import HttpResponse
+from django.http import HttpResponse, JsonResponse
 from django.shortcuts import redirect
-from ninja import Field, Form, Router, Schema, Query
+from ninja import Field, Form, Query, Router, Schema
 from ninja.security import SessionAuth
 
 from utilities.chem import FormatConverter
 from utilities.misc import PackageExporter
 
-from .logic import GroupManager, PackageManager, SettingManager, UserManager, SearchManager
+from .logic import GroupManager, PackageManager, SearchManager, SettingManager, UserManager
 from .models import (
     Compound,
     CompoundStructure,
     Edge,
+    EnviFormer,
     EPModel,
+    Group,
+    GroupPackagePermission,
+    MLRelativeReasoning,
     Node,
+    PackageBasedModel,
+    ParallelRule,
     Pathway,
     Reaction,
     Rule,
+    RuleBasedRelativeReasoning,
     Scenario,
     SimpleAmbitRule,
     User,
     UserPackagePermission,
-    ParallelRule,
 )
 
 Package = s.GET_PACKAGE_MODEL()
 
 
+def get_package_for_write(user, package_uuid):
+    p = PackageManager.get_package_by_id(user, package_uuid)
+    if not PackageManager.writable(user, p):
+        raise ValueError("You do not have the rights to write to this Package!")
+    return p
+
+
 def _anonymous_or_real(request):
     if request.user.is_authenticated and not request.user.is_anonymous:
         return request.user
@@ -81,6 +94,8 @@ class SimpleObject(Schema):
             return "reviewed" if obj.compound.package.reviewed else "unreviewed"
         elif isinstance(obj, Node) or isinstance(obj, Edge):
             return "reviewed" if obj.pathway.package.reviewed else "unreviewed"
+        elif isinstance(obj, dict) and "review_status" in obj:
+            return "reviewed" if obj.get("review_status") else "unreviewed"
         else:
             raise ValueError("Object has no package")
 
@@ -200,6 +215,82 @@ def get_user(request, user_uuid):
         }
 
 
+########
+# Group #
+########
+
+
+class GroupMember(Schema):
+    id: str
+    identifier: str
+    name: str
+
+
+class GroupWrapper(Schema):
+    group: List[SimpleGroup]
+
+
+class GroupSchema(Schema):
+    description: str
+    id: str = Field(None, alias="url")
+    identifier: str = "group"
+    members: List[GroupMember] = Field([], alias="members")
+    name: str = Field(None, alias="name")
+    ownerid: str = Field(None, alias="owner.url")
+    ownername: str = Field(None, alias="owner.get_name")
+    packages: List["SimplePackage"] = Field([], alias="packages")
+    readers: List[GroupMember] = Field([], alias="readers")
+    writers: List[GroupMember] = Field([], alias="writers")
+
+    @staticmethod
+    def resolve_members(obj: Group):
+        res = []
+        for member in obj.user_member.all():
+            res.append(GroupMember(id=member.url, identifier="usermember", name=member.get_name()))
+
+        for member in obj.group_member.all():
+            res.append(GroupMember(id=member.url, identifier="groupmember", name=member.get_name()))
+
+        return res
+
+    @staticmethod
+    def resolve_packages(obj: Group):
+        return Package.objects.filter(
+            id__in=[
+                GroupPackagePermission.objects.filter(group=obj).values_list(
+                    "package_id", flat=True
+                )
+            ]
+        )
+
+    @staticmethod
+    def resolve_readers(obj: Group):
+        return GroupSchema.resolve_members(obj)
+
+    @staticmethod
+    def resolve_writers(obj: Group):
+        return [GroupMember(id=obj.owner.url, identifier="usermember", name=obj.owner.username)]
+
+
+@router.get("/group", response={200: GroupWrapper, 403: Error})
+def get_groups(request):
+    return {"group": GroupManager.get_groups(request.user)}
+
+
+@router.get("/group/{uuid:group_uuid}", response={200: GroupSchema, 403: Error})
+def get_group(request, group_uuid):
+    try:
+        g = GroupManager.get_group_by_id(request.user, group_uuid)
+        return g
+    except ValueError:
+        return 403, {
+            "message": f"Getting Group with id {group_uuid} failed due to insufficient rights!"
+        }
+
+
+##########
+# Search #
+##########
 class Search(Schema):
     packages: List[str] = Field(alias="packages[]")
     search: str
@@ -237,11 +328,11 @@ def search(request, search: Query[Search]):
         if "Compound Structures" in search_res:
             res["structure"] = search_res["Compound Structures"]
 
-        if "Reaction" in search_res:
-            res["reaction"] = search_res["Reaction"]
+        if "Reactions" in search_res:
+            res["reaction"] = search_res["Reactions"]
 
-        if "Pathway" in search_res:
-            res["pathway"] = search_res["Pathway"]
+        if "Pathways" in search_res:
+            res["pathway"] = search_res["Pathways"]
 
         if "Rules" in search_res:
             res["rule"] = search_res["Rules"]
@@ -292,7 +383,7 @@ class PackageSchema(Schema):
             ).values_list("user", flat=True)
         ).distinct()
 
-        return [{u.id: u.name} for u in users]
+        return [{u.id: u.get_name()} for u in users]
 
     @staticmethod
     def resolve_writers(obj: Package):
@@ -302,7 +393,7 @@ class PackageSchema(Schema):
             ).values_list("user", flat=True)
         ).distinct()
 
-        return [{u.id: u.name} for u in users]
+        return [{u.id: u.get_name()} for u in users]
 
     @staticmethod
     def resolve_review_comment(obj):
@@ -373,7 +464,7 @@ class UpdatePackage(Schema):
 @router.post("/package/{uuid:package_uuid}", response={200: PackageSchema | Any, 400: Error})
 def update_package(request, package_uuid, pack: Form[UpdatePackage]):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
         if pack.hiddenMethod:
             if pack.hiddenMethod == "DELETE":
@@ -469,21 +560,42 @@ class CompoundSchema(Schema):
 
     @staticmethod
     def resolve_halflifes(obj: Compound):
-        return []
+        res = []
+        for scen, hls in obj.half_lifes().items():
+            for hl in hls:
+                res.append(
+                    {
+                        "hl": str(hl.dt50),
+                        "hlComment": hl.comment,
+                        "hlFit": hl.fit,
+                        "hlModel": hl.model,
+                        "scenarioId": scen.url,
+                        "scenarioName": scen.name,
+                        "scenarioType": scen.scenario_type,
+                        "source": hl.source,
+                    }
+                )
+        return res
 
     @staticmethod
     def resolve_pubchem_compound_references(obj: Compound):
+        # TODO
         return []
 
     @staticmethod
     def resolve_pathway_scenarios(obj: Compound):
-        return [
-            {
-                "scenarioId": "https://envipath.org/package/5882df9c-dae1-4d80-a40e-db4724271456/scenario/cd8350cd-4249-4111-ba9f-4e2209338501",
-                "scenarioName": "Fritz, R. & Brauner, A. (1989) - (00004)",
-                "scenarioType": "Soil",
-            }
-        ]
+        res = []
+        for pw in obj.related_pathways:
+            for scen in pw.scenarios.all():
+                res.append(
+                    {
+                        "scenarioId": scen.url,
+                        "scenarioName": scen.name,
+                        "scenarioType": scen.scenario_type,
+                    }
+                )
+
+        return res
 
 
 class CompoundStructureSchema(Schema):
@@ -536,7 +648,22 @@ class CompoundStructureSchema(Schema):
 
     @staticmethod
     def resolve_halflifes(obj: CompoundStructure):
-        return []
+        res = []
+        for scen, hls in obj.half_lifes().items():
+            for hl in hls:
+                res.append(
+                    {
+                        "hl": str(hl.dt50),
+                        "hlComment": hl.comment,
+                        "hlFit": hl.fit,
+                        "hlModel": hl.model,
+                        "scenarioId": scen.url,
+                        "scenarioName": scen.name,
+                        "scenarioType": scen.scenario_type,
+                        "source": hl.source,
+                    }
+                )
+        return res
 
     @staticmethod
     def resolve_pubchem_compound_references(obj: CompoundStructure):
@@ -544,13 +671,18 @@ class CompoundStructureSchema(Schema):
 
     @staticmethod
     def resolve_pathway_scenarios(obj: CompoundStructure):
-        return [
-            {
-                "scenarioId": "https://envipath.org/package/5882df9c-dae1-4d80-a40e-db4724271456/scenario/cd8350cd-4249-4111-ba9f-4e2209338501",
-                "scenarioName": "Fritz, R. & Brauner, A. (1989) - (00004)",
-                "scenarioType": "Soil",
-            }
-        ]
+        res = []
+        for pw in obj.related_pathways:
+            for scen in pw.scenarios.all():
+                res.append(
+                    {
+                        "scenarioId": scen.url,
+                        "scenarioName": scen.name,
+                        "scenarioType": scen.scenario_type,
+                    }
+                )
+
+        return res
 
 
 class CompoundStructureWrapper(Schema):
@@ -635,7 +767,7 @@ def create_package_compound(
     c: Form[CreateCompound],
 ):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
         # inchi is not used atm
         c = Compound.create(
             p, c.compoundSmiles, c.compoundName, c.compoundDescription, inchi=c.inchi
@@ -648,14 +780,10 @@ def create_package_compound(
 @router.delete("/package/{uuid:package_uuid}/compound/{uuid:compound_uuid}")
 def delete_compound(request, package_uuid, compound_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
-
-        if PackageManager.writable(request.user, p):
-            c = Compound.objects.get(package=p, uuid=compound_uuid)
-            c.delete()
-            return redirect(f"{p.url}/compound")
-        else:
-            raise ValueError("You do not have the rights to delete this Compound!")
+        p = get_package_for_write(request.user, package_uuid)
+        c = Compound.objects.get(package=p, uuid=compound_uuid)
+        c.delete()
+        return redirect(f"{p.url}/compound")
     except ValueError:
         return 403, {
             "message": f"Deleting Compound with id {compound_uuid} failed due to insufficient rights!"
@@ -667,31 +795,29 @@ def delete_compound(request, package_uuid, compound_uuid):
 )
 def delete_compound_structure(request, package_uuid, compound_uuid, structure_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
-        if PackageManager.writable(request.user, p):
-            c = Compound.objects.get(package=p, uuid=compound_uuid)
-            cs = CompoundStructure.objects.get(compound=c, uuid=structure_uuid)
+        c = Compound.objects.get(package=p, uuid=compound_uuid)
+        cs = CompoundStructure.objects.get(compound=c, uuid=structure_uuid)
 
-            # Check if we have to delete the compound as no structure is left
-            if len(cs.compound.structures.all()) == 1:
-                # This will delete the structure as well
+        # Check if we have to delete the compound as no structure is left
+        if len(cs.compound.structures.all()) == 1:
+            # This will delete the structure as well
+            c.delete()
+            return redirect(p.url + "/compound")
+        else:
+            if cs.normalized_structure:
                 c.delete()
                 return redirect(p.url + "/compound")
             else:
-                if cs.normalized_structure:
-                    c.delete()
-                    return redirect(p.url + "/compound")
+                if c.default_structure == cs:
+                    cs.delete()
+                    c.default_structure = c.structures.all().first()
+                    return redirect(c.url + "/structure")
                 else:
-                    if c.default_structure == cs:
-                        cs.delete()
-                        c.default_structure = c.structures.all().first()
-                        return redirect(c.url + "/structure")
-                    else:
-                        cs.delete()
-                        return redirect(c.url + "/structure")
-        else:
-            raise ValueError("You do not have the rights to delete this CompoundStructure!")
+                    cs.delete()
+                    return redirect(c.url + "/structure")
+
     except ValueError:
         return 403, {
             "message": f"Deleting CompoundStructure with id {compound_uuid} failed due to insufficient rights!"
@@ -878,13 +1004,18 @@ def create_package_simple_rule(
     r: Form[CreateSimpleRule],
 ):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
         if r.rdkitrule and r.rdkitrule.strip() == "true":
             raise ValueError("Not yet implemented!")
         else:
             sr = SimpleAmbitRule.create(
-                p, r.name, r.description, r.smirks, r.reactantFilterSmarts, r.productFilterSmarts
+                p,
+                r.name,
+                r.description,
+                r.smirks,
+                r.reactantFilterSmarts,
+                r.productFilterSmarts,
             )
 
         return redirect(sr.url)
@@ -909,7 +1040,7 @@ def create_package_parallel_rule(
     r: Form[CreateParallelRule],
 ):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
         srs = SimpleRule.objects.filter(package=p, url__in=r.simpleRules)
 
@@ -953,7 +1084,7 @@ def post_package_parallel_rule(request, package_uuid, rule_uuid, compound: Form[
 
 def _post_package_rule(request, package_uuid, rule_uuid, compound: Form[str]):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
         r = Rule.objects.get(package=p, uuid=rule_uuid)
 
         if compound is not None:
@@ -998,14 +1129,11 @@ def delete_parallel_rule(request, package_uuid, rule_uuid):
 
 def _delete_rule(request, package_uuid, rule_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
+        r = Rule.objects.get(package=p, uuid=rule_uuid)
+        r.delete()
+        return redirect(f"{p.url}/rule")
 
-        if PackageManager.writable(request.user, p):
-            r = Rule.objects.get(package=p, uuid=rule_uuid)
-            r.delete()
-            return redirect(f"{p.url}/rule")
-        else:
-            raise ValueError("You do not have the rights to delete this Rule!")
     except ValueError:
         return 403, {
             "message": f"Deleting Rule with id {rule_uuid} failed due to insufficient rights!"
@@ -1037,7 +1165,7 @@ class ReactionSchema(Schema):
     name: str = Field(None, alias="name")
     pathways: List["SimplePathway"] = Field([], alias="related_pathways")
     products: List["ReactionCompoundStructure"] = Field([], alias="products")
-    references: List[Dict[str, List[str]]] = Field([], alias="references")
+    references: Dict[str, List[str]] = Field({}, alias="references")
     reviewStatus: str = Field(None, alias="review_status")
     scenarios: List["SimpleScenario"] = Field([], alias="scenarios")
     smirks: str = Field("", alias="smirks")
@@ -1053,8 +1181,12 @@ class ReactionSchema(Schema):
 
     @staticmethod
     def resolve_references(obj: Reaction):
-        # TODO
-        return []
+        rhea_refs = []
+        for rhea in obj.get_rhea_identifiers():
+            rhea_refs.append(f"{rhea.identifier_value}")
+
+        # TODO UniProt
+        return {"rheaReferences": rhea_refs, "uniprotCount": []}
 
     @staticmethod
     def resolve_medline_references(obj: Reaction):
@@ -1116,7 +1248,7 @@ def create_package_reaction(
     r: Form[CreateReaction],
 ):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
         if r.smirks is None and (r.educt is None or r.product is None):
             raise ValueError("Either SMIRKS or educt/product must be provided")
@@ -1162,14 +1294,11 @@ def create_package_reaction(
 @router.delete("/package/{uuid:package_uuid}/reaction/{uuid:reaction_uuid}")
 def delete_reaction(request, package_uuid, reaction_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
-        if PackageManager.writable(request.user, p):
-            r = Reaction.objects.get(package=p, uuid=reaction_uuid)
-            r.delete()
-            return redirect(f"{p.url}/reaction")
-        else:
-            raise ValueError("You do not have the rights to delete this Reaction!")
+        r = Reaction.objects.get(package=p, uuid=reaction_uuid)
+        r.delete()
+        return redirect(f"{p.url}/reaction")
     except ValueError:
         return 403, {
             "message": f"Deleting Reaction with id {reaction_uuid} failed due to insufficient rights!"
@@ -1241,17 +1370,53 @@ def get_package_scenario(request, package_uuid, scenario_uuid):
         }
 
 
-@router.delete("/package/{uuid:package_uuid}/scenario")
-def delete_scenarios(request, package_uuid, scenario_uuid):
-    try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+@router.post("/package/{uuid:package_uuid}/scenario", response={200: str | Any, 403: Error})
+def create_package_scenario(request, package_uuid):
+    from utilities.legacy import build_additional_information_from_request
+
+    try:
+        p = get_package_for_write(request.user, package_uuid)
+
+        scen_date = None
+        date_year = request.POST.get("dateYear")
+        date_month = request.POST.get("dateMonth")
+        date_day = request.POST.get("dateDay")
+
+        if date_year:
+            scen_date = date_year
+            if date_month:
+                scen_date += f"-{date_month}"
+                if date_day:
+                    scen_date += f"-{date_day}"
+
+        name = request.POST.get("studyname")
+        description = request.POST.get("studydescription")
+        study_type = request.POST.get("type")
+
+        ais = []
+        types = request.POST.get("adInfoTypes[]", "").split(",")
+        for t in types:
+            ais.append(build_additional_information_from_request(request, t))
+
+        new_s = Scenario.create(p, name, description, scen_date, study_type, ais)
+
+        return JsonResponse({"scenarioLocation": new_s.url})
+
+    except ValueError:
+        return 403, {
+            "message": f"Getting Package with id {package_uuid} failed due to insufficient rights!"
+        }
+
+
+@router.delete("/package/{uuid:package_uuid}/scenario")
+def delete_scenarios(request, package_uuid):
+    try:
+        p = get_package_for_write(request.user, package_uuid)
+
+        scens = Scenario.objects.filter(package=p)
+        scens.delete()
+        return redirect(f"{p.url}/scenario")
 
-        if PackageManager.writable(request.user, p):
-            scens = Scenario.objects.filter(package=p)
-            scens.delete()
-            return redirect(f"{p.url}/scenario")
-        else:
-            raise ValueError("You do not have the rights to delete Scenarios!")
     except ValueError:
         return 403, {"message": "Deleting Scenarios failed due to insufficient rights!"}
 
@@ -1259,14 +1424,12 @@ def delete_scenarios(request, package_uuid, scenario_uuid):
 @router.delete("/package/{uuid:package_uuid}/scenario/{uuid:scenario_uuid}")
 def delete_scenario(request, package_uuid, scenario_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
+
+        scen = Scenario.objects.get(package=p, uuid=scenario_uuid)
+        scen.delete()
+        return redirect(f"{p.url}/scenario")
 
-        if PackageManager.writable(request.user, p):
-            scen = Scenario.objects.get(package=p, uuid=scenario_uuid)
-            scen.delete()
-            return redirect(f"{p.url}/scenario")
-        else:
-            raise ValueError("You do not have the rights to delete this Scenario!")
     except ValueError:
         return 403, {
             "message": f"Deleting Scenario with id {scenario_uuid} failed due to insufficient rights!"
@@ -1289,8 +1452,8 @@ class PathwayEdge(Schema):
     pseudo: bool = False
     rule: Optional[str] = Field(None, alias="rule")
     scenarios: List[SimpleScenario] = Field([], alias="scenarios")
-    source: int = -1
-    target: int = -1
+    source: int = Field(-1)
+    target: int = Field(-1)
 
     @staticmethod
     def resolve_rule(obj: Edge):
@@ -1303,7 +1466,7 @@ class PathwayEdge(Schema):
 
 class PathwayNode(Schema):
     atomCount: int = Field(None, alias="atom_count")
-    depth: int = Field(None, alias="depth")
+    depth: float = Field(None, alias="depth")
     dt50s: List[Dict[str, str]] = Field([], alias="dt50s")
     engineeredIntermediate: bool = Field(None, alias="engineered_intermediate")
     id: str = Field(None, alias="url")
@@ -1353,9 +1516,9 @@ class PathwaySchema(Schema):
     isIncremental: bool = Field(None, alias="is_incremental")
     isPredicted: bool = Field(None, alias="is_predicted")
     lastModified: int = Field(None, alias="last_modified")
-    links: List[PathwayEdge] = Field([], alias="edges")
+    links: List[PathwayEdge] = Field([])
     name: str = Field(None, alias="name")
-    nodes: List[PathwayNode] = Field([], alias="nodes")
+    nodes: List[PathwayNode] = Field([])
     pathwayName: str = Field(None, alias="name")
     reviewStatus: str = Field(None, alias="review_status")
     scenarios: List["SimpleScenario"] = Field([], alias="scenarios")
@@ -1377,6 +1540,14 @@ class PathwaySchema(Schema):
     def resolve_last_modified(obj: Pathway):
         return int(obj.modified.timestamp())
 
+    @staticmethod
+    def resolve_links(obj: Pathway):
+        return obj.d3_json().get("links", [])
+
+    @staticmethod
+    def resolve_nodes(obj: Pathway):
+        return obj.d3_json().get("nodes", [])
+
 
 @router.get("/pathway", response={200: PathwayWrapper, 403: Error})
 def get_pathways(request):
@@ -1420,16 +1591,16 @@ class CreatePathway(Schema):
     selectedSetting: str | None = None
 
 
-@router.post("/package/{uuid:package_uuid}/pathway")
-def create_pathway(
+@router.post("/package/{uuid:package_uuid}/pathway", response={200: Any, 403: Error})
+def create_package_pathway(
     request,
     package_uuid,
     pw: Form[CreatePathway],
 ):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
-        stand_smiles = FormatConverter.standardize(pw.smilesinput.strip())
+        stand_smiles = FormatConverter.standardize(pw.smilesinput.strip(), remove_stereo=True)
 
         new_pw = Pathway.create(p, stand_smiles, name=pw.name, description=pw.description)
 
@@ -1447,6 +1618,7 @@ def create_pathway(
                 setting = SettingManager.get_setting_by_url(request.user, pw.selectedSetting)
 
             new_pw.setting = setting
+            new_pw.kv.update({"status": "running"})
             new_pw.save()
 
             from .tasks import dispatch, predict
@@ -1455,20 +1627,18 @@ def create_pathway(
 
         return redirect(new_pw.url)
     except ValueError as e:
-        return 400, {"message": str(e)}
+        return 403, {"message": str(e)}
 
 
 @router.delete("/package/{uuid:package_uuid}/pathway/{uuid:pathway_uuid}")
 def delete_pathway(request, package_uuid, pathway_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
+
+        pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
+        pw.delete()
+        return redirect(f"{p.url}/pathway")
 
-        if PackageManager.writable(request.user, p):
-            pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
-            pw.delete()
-            return redirect(f"{p.url}/pathway")
-        else:
-            raise ValueError("You do not have the rights to delete this pathway!")
     except ValueError:
         return 403, {
             "message": f"Deleting Pathway with id {pathway_uuid} failed due to insufficient rights!"
@@ -1576,7 +1746,7 @@ class CreateNode(Schema):
 )
 def add_pathway_node(request, package_uuid, pathway_uuid, n: Form[CreateNode]):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
         pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
 
         if n.nodeDepth is not None and n.nodeDepth.strip() != "":
@@ -1594,15 +1764,13 @@ def add_pathway_node(request, package_uuid, pathway_uuid, n: Form[CreateNode]):
 @router.delete("/package/{uuid:package_uuid}/pathway/{uuid:pathway_uuid}/node/{uuid:node_uuid}")
 def delete_node(request, package_uuid, pathway_uuid, node_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
+
+        pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
+        n = Node.objects.get(pathway=pw, uuid=node_uuid)
+        n.delete()
+        return redirect(f"{pw.url}/node")
 
-        if PackageManager.writable(request.user, p):
-            pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
-            n = Node.objects.get(pathway=pw, uuid=node_uuid)
-            n.delete()
-            return redirect(f"{pw.url}/node")
-        else:
-            raise ValueError("You do not have the rights to delete this Node!")
     except ValueError:
         return 403, {
             "message": f"Deleting Node with id {node_uuid} failed due to insufficient rights!"
@@ -1632,14 +1800,14 @@ class EdgeSchema(Schema):
     id: str = Field(None, alias="url")
     identifier: str = "edge"
     name: str = Field(None, alias="name")
-    reactionName: str = Field(None, alias="edge_label.name")
+    reactionName: str = Field(None, alias="edge_label.get_name")
     reactionURI: str = Field(None, alias="edge_label.url")
     reviewStatus: str = Field(None, alias="review_status")
     scenarios: List["SimpleScenario"] = Field([], alias="scenarios")
     startNodes: List["EdgeNode"] = Field([], alias="start_nodes")
 
     @staticmethod
-    def resolve_review_status(obj: Node):
+    def resolve_review_status(obj: Edge):
         return "reviewed" if obj.pathway.package.reviewed else "unreviewed"
 
 
@@ -1681,12 +1849,12 @@ class CreateEdge(Schema):
 
 
 @router.post(
-    "/package/{uuid:package_uuid}/üathway/{uuid:pathway_uuid}/edge",
+    "/package/{uuid:package_uuid}/pathway/{uuid:pathway_uuid}/edge",
     response={200: str | Any, 403: Error},
 )
 def add_pathway_edge(request, package_uuid, pathway_uuid, e: Form[CreateEdge]):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
         pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
 
         if e.edgeAsSmirks is None and (e.educts is None or e.products is None):
@@ -1700,10 +1868,26 @@ def add_pathway_edge(request, package_uuid, pathway_uuid, e: Form[CreateEdge]):
 
         if e.edgeAsSmirks:
             for ed in e.edgeAsSmirks.split(">>")[0].split("\\."):
-                educts.append(Node.objects.get(pathway=pw, default_node_label__smiles=ed))
+                stand_ed = FormatConverter.standardize(ed, remove_stereo=True)
+                educts.append(
+                    Node.objects.get(
+                        pathway=pw,
+                        default_node_label=CompoundStructure.objects.get(
+                            compound__package=p, smiles=stand_ed
+                        ).compound.default_structure,
+                    )
+                )
 
             for pr in e.edgeAsSmirks.split(">>")[1].split("\\."):
-                products.append(Node.objects.get(pathway=pw, default_node_label__smiles=pr))
+                stand_pr = FormatConverter.standardize(pr, remove_stereo=True)
+                products.append(
+                    Node.objects.get(
+                        pathway=pw,
+                        default_node_label=CompoundStructure.objects.get(
+                            compound__package=p, smiles=stand_pr
+                        ).compound.default_structure,
+                    )
+                )
         else:
             for ed in e.educts.split(","):
                 educts.append(Node.objects.get(pathway=pw, url=ed.strip()))
@@ -1716,7 +1900,7 @@ def add_pathway_edge(request, package_uuid, pathway_uuid, e: Form[CreateEdge]):
             start_nodes=educts,
             end_nodes=products,
             rule=None,
-            name=e.name,
+            name=None,
             description=e.edgeReason,
         )
 
@@ -1728,15 +1912,13 @@ def add_pathway_edge(request, package_uuid, pathway_uuid, e: Form[CreateEdge]):
 @router.delete("/package/{uuid:package_uuid}/pathway/{uuid:pathway_uuid}/edge/{uuid:edge_uuid}")
 def delete_edge(request, package_uuid, pathway_uuid, edge_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
+
+        pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
+        e = Edge.objects.get(pathway=pw, uuid=edge_uuid)
+        e.delete()
+        return redirect(f"{pw.url}/edge")
 
-        if PackageManager.writable(request.user, p):
-            pw = Pathway.objects.get(package=p, uuid=pathway_uuid)
-            e = Edge.objects.get(pathway=pw, uuid=edge_uuid)
-            e.delete()
-            return redirect(f"{pw.url}/edge")
-        else:
-            raise ValueError("You do not have the rights to delete this Edge!")
     except ValueError:
         return 403, {
             "message": f"Deleting Edge with id {edge_uuid} failed due to insufficient rights!"
@@ -1753,26 +1935,46 @@ class ModelWrapper(Schema):
 class ModelSchema(Schema):
     aliases: List[str] = Field([], alias="aliases")
     description: str = Field(None, alias="description")
-    evalPackages: List["SimplePackage"] = Field([])
+    evalPackages: List["SimplePackage"] = Field([], alias="eval_packages")
     id: str = Field(None, alias="url")
     identifier: str = "relative-reasoning"
-    #   "info" : {
-    #       "Accuracy (Single-Gen)" : "0.5932962678936605" ,
-    #       "Area under PR-Curve (Single-Gen)" : "0.5654653182134282" ,
-    #       "Area under ROC-Curve (Single-Gen)" : "0.8178302405034772" ,
-    #       "Precision (Single-Gen)" : "0.6978730822873083" ,
-    #       "Probability Threshold" : "0.5" ,
-    #       "Recall/Sensitivity (Single-Gen)" : "0.4484149210261006"
-    #     } ,
+    info: dict = Field({}, alias="info")
     name: str = Field(None, alias="name")
-    pathwayPackages: List["SimplePackage"] = Field([])
+    pathwayPackages: List["SimplePackage"] = Field([], alias="pathway_packages")
     reviewStatus: str = Field(None, alias="review_status")
-    rulePackages: List["SimplePackage"] = Field([])
+    rulePackages: List["SimplePackage"] = Field([], alias="rule_packages")
     scenarios: List["SimpleScenario"] = Field([], alias="scenarios")
-    status: str
-    statusMessage: str
-    threshold: str
-    type: str
+    status: str = Field(None, alias="model_status")
+    statusMessage: str = Field(None, alias="status_message")
+    threshold: str = Field(None, alias="threshold")
+    type: str = Field(None, alias="model_type")
+
+    @staticmethod
+    def resolve_info(obj: EPModel):
+        return {}
+
+    @staticmethod
+    def resolve_status_message(obj: EPModel):
+        for k, v in PackageBasedModel.PROGRESS_STATUS_CHOICES.items():
+            if k == obj.model_status:
+                return v
+
+        return None
+
+    @staticmethod
+    def resolve_threshold(obj: EPModel):
+        return f"{obj.threshold:.2f}"
+
+    @staticmethod
+    def resolve_model_type(obj: EPModel):
+        if isinstance(obj, RuleBasedRelativeReasoning):
+            return "RULEBASED"
+        elif isinstance(obj, MLRelativeReasoning):
+            return "ECC"
+        elif isinstance(obj, EnviFormer):
+            return "ENVIFORMER"
+        else:
+            return None
 
 
 @router.get("/model", response={200: ModelWrapper, 403: Error})
@@ -1809,7 +2011,7 @@ def get_model(request, package_uuid, model_uuid, c: Query[Classify]):
                 return 400, {"message": "Received empty SMILES"}
 
             try:
-                stand_smiles = FormatConverter.standardize(c.smiles)
+                stand_smiles = FormatConverter.standardize(c.smiles, remove_stereo=True)
             except ValueError:
                 return 400, {"message": f'"{c.smiles}" is not a valid SMILES'}
 
@@ -1833,7 +2035,7 @@ def get_model(request, package_uuid, model_uuid, c: Query[Classify]):
                     if pr.rule:
                         res["id"] = pr.rule.url
                         res["identifier"] = pr.rule.get_rule_identifier()
-                        res["name"] = pr.rule.name
+                        res["name"] = pr.rule.get_name()
                         res["reviewStatus"] = (
                             "reviewed" if pr.rule.package.reviewed else "unreviewed"
                         )
@@ -1852,14 +2054,11 @@ def get_model(request, package_uuid, model_uuid, c: Query[Classify]):
 @router.delete("/package/{uuid:package_uuid}/model/{uuid:model_uuid}")
 def delete_model(request, package_uuid, model_uuid):
     try:
-        p = PackageManager.get_package_by_id(request.user, package_uuid)
+        p = get_package_for_write(request.user, package_uuid)
 
-        if PackageManager.writable(request.user, p):
-            m = EPModel.objects.get(package=p, uuid=model_uuid)
-            m.delete()
-            return redirect(f"{p.url}/model")
-        else:
-            raise ValueError("You do not have the rights to delete this Model!")
+        m = EPModel.objects.get(package=p, uuid=model_uuid)
+        m.delete()
+        return redirect(f"{p.url}/model")
     except ValueError:
         return 403, {
             "message": f"Deleting Model with id {model_uuid} failed due to insufficient rights!"

epdb/logic.py

@@ -1,4 +1,3 @@
-import json
 import logging
 import re
 from typing import Any, Dict, List, Optional, Set, Union, Tuple
@@ -11,6 +10,7 @@ from django.db import transaction
 from pydantic import ValidationError
 
 from epdb.models import (
+    AdditionalInformation,
     Compound,
     CompoundStructure,
     Edge,
@@ -22,6 +22,7 @@ from epdb.models import (
     Node,
     Pathway,
     Permission,
+    PropertyPluginModel,
     Reaction,
     Rule,
     Setting,
@@ -194,8 +195,6 @@ class UserManager(object):
         if clean_username != username or clean_email != email:
             # This will be caught by the try in view.py/register
             raise ValueError("Invalid username or password")
-        # avoid circular import :S
-        from .tasks import send_registration_mail
 
         extra_fields = {"is_active": not s.ADMIN_APPROVAL_REQUIRED}
 
@@ -214,10 +213,6 @@ class UserManager(object):
         u.default_package = p
         u.save()
 
-        if not u.is_active:
-            # send email for verification
-            send_registration_mail.delay(u.pk)
-
         if set_setting:
             u.default_setting = Setting.objects.get(global_default=True)
             u.save()
@@ -639,15 +634,30 @@ class PackageManager(object):
 
         # Stores old_id to new_id
         mapping = {}
-        # Stores new_scen_id to old_parent_scen_id
-        parent_mapping = {}
         # Mapping old scen_id to old_obj_id
         scen_mapping = defaultdict(list)
         # Enzymelink Mapping rule_id to enzymelink objects
         enzyme_mapping = defaultdict(list)
 
+        # old_parent_id to child
+        postponed_scens = defaultdict(list)
+
         # Store Scenarios
         for scenario in data["scenarios"]:
+            skip_scen = False
+            # Check if parent exists and park this Scenario to convert it later into an
+            # AdditionalInformation object
+            for ex in scenario.get("additionalInformationCollection", {}).get(
+                "additionalInformation", []
+            ):
+                if ex["name"] == "referringscenario":
+                    postponed_scens[ex["data"]].append(scenario)
+                    skip_scen = True
+                    break
+
+            if skip_scen:
+                continue
+
             scen = Scenario()
             scen.package = pack
             scen.uuid = UUID(scenario["id"].split("/")[-1]) if keep_ids else uuid4()
@@ -660,19 +670,12 @@ class PackageManager(object):
 
             mapping[scenario["id"]] = scen.uuid
 
-            new_add_inf = defaultdict(list)
-            # TODO Store AI...
             for ex in scenario.get("additionalInformationCollection", {}).get(
                 "additionalInformation", []
             ):
                 name = ex["name"]
                 addinf_data = ex["data"]
 
-                # park the parent scen id for now and link it later
-                if name == "referringscenario":
-                    parent_mapping[scen.uuid] = addinf_data
-                    continue
-
                 # Broken eP Data
                 if name == "initialmasssediment" and addinf_data == "missing data":
                     continue
@@ -680,17 +683,11 @@ class PackageManager(object):
                     continue
 
                 try:
-                    res = AdditionalInformationConverter.convert(name, addinf_data)
-                    res_cls_name = res.__class__.__name__
-                    ai_data = json.loads(res.model_dump_json())
-                    ai_data["uuid"] = f"{uuid4()}"
-                    new_add_inf[res_cls_name].append(ai_data)
-                except ValidationError:
+                    ai = AdditionalInformationConverter.convert(name, addinf_data)
+                    AdditionalInformation.create(pack, ai, scenario=scen)
+                except (ValidationError, ValueError):
                     logger.error(f"Failed to convert {name} with {addinf_data}")
 
-            scen.additional_information = new_add_inf
-            scen.save()
-
         print("Scenarios imported...")
 
         # Store compounds and its structures
@@ -930,14 +927,46 @@ class PackageManager(object):
 
         print("Pathways imported...")
 
-        # Linking Phase
-        for child, parent in parent_mapping.items():
-            child_obj = Scenario.objects.get(uuid=child)
-            parent_obj = Scenario.objects.get(uuid=mapping[parent])
-            child_obj.parent = parent_obj
-            child_obj.save()
+        for parent, children in postponed_scens.items():
+            for child in children:
+                for ex in child.get("additionalInformationCollection", {}).get(
+                    "additionalInformation", []
+                ):
+                    child_id = child["id"]
+                    name = ex["name"]
+                    addinf_data = ex["data"]
+
+                    if name == "referringscenario":
+                        continue
+                    # Broken eP Data
+                    if name == "initialmasssediment" and addinf_data == "missing data":
+                        continue
+                    if name == "columnheight" and addinf_data == "(2)-(2.5);(6)-(8)":
+                        continue
+
+                    ai = AdditionalInformationConverter.convert(name, addinf_data)
+
+                    if child_id not in scen_mapping:
+                        logger.info(
+                            f"{child_id} not found in scen_mapping. Seems like its not attached to any object"
+                        )
+                        print(
+                            f"{child_id} not found in scen_mapping. Seems like its not attached to any object"
+                        )
+
+                    scen = Scenario.objects.get(uuid=mapping[parent])
+                    mapping[child_id] = scen.uuid
+                    for obj in scen_mapping[child_id]:
+                        _ = AdditionalInformation.create(pack, ai, scen, content_object=obj)
 
         for scen_id, objects in scen_mapping.items():
+            new_id = mapping.get(scen_id)
+
+            if new_id is None:
+                logger.warning(f"Could not find mapping for {scen_id}")
+                print(f"Could not find mapping for {scen_id}")
+                continue
+
             scen = Scenario.objects.get(uuid=mapping[scen_id])
             for o in objects:
                 o.scenarios.add(scen)
@@ -970,6 +999,7 @@ class PackageManager(object):
                     matches = re.findall(r">(R[0-9]+)<", evidence["evidence"])
                     if not matches or len(matches) != 1:
                         logger.warning(f"Could not find reaction id in {evidence['evidence']}")
+                        print(f"Could not find reaction id in {evidence['evidence']}")
                         continue
 
                     e.add_kegg_reaction_id(matches[0])
@@ -989,7 +1019,6 @@ class PackageManager(object):
         print("Fixing Node depths...")
         total_pws = Pathway.objects.filter(package=pack).count()
         for p, pw in enumerate(Pathway.objects.filter(package=pack)):
-            print(pw.url)
             in_count = defaultdict(lambda: 0)
             out_count = defaultdict(lambda: 0)
 
@@ -1025,7 +1054,6 @@ class PackageManager(object):
                             if str(prod.uuid) not in seen:
                                 old_depth = prod.depth
                                 if old_depth != i + 1:
-                                    print(f"updating depth from {old_depth} to {i + 1}")
                                     prod.depth = i + 1
                                     prod.save()
 
@@ -1036,7 +1064,7 @@ class PackageManager(object):
                 if new_level:
                     levels.append(new_level)
 
-            print(f"{p + 1}/{total_pws} fixed.")
+            print(f"{p + 1}/{total_pws} fixed.", end="\r")
 
         return pack
 
@@ -1115,19 +1143,23 @@ class SettingManager(object):
         description: str = None,
         max_nodes: int = None,
         max_depth: int = None,
-        rule_packages: List[Package] = None,
+        rule_packages: List[Package] | None = None,
         model: EPModel = None,
         model_threshold: float = None,
         expansion_scheme: ExpansionSchemeChoice = ExpansionSchemeChoice.BFS,
+        property_models: List["PropertyPluginModel"] | None = None,
     ):
         new_s = Setting()
+
         # Clean for potential XSS
         new_s.name = nh3.clean(name, tags=s.ALLOWED_HTML_TAGS).strip()
         new_s.description = nh3.clean(description, tags=s.ALLOWED_HTML_TAGS).strip()
+
         new_s.max_nodes = max_nodes
         new_s.max_depth = max_depth
         new_s.model = model
         new_s.model_threshold = model_threshold
+        new_s.expansion_scheme = expansion_scheme
 
         new_s.save()
 
@@ -1136,6 +1168,11 @@ class SettingManager(object):
                 new_s.rule_packages.add(r)
             new_s.save()
 
+        if property_models is not None:
+            for pm in property_models:
+                new_s.property_models.add(pm)
+            new_s.save()
+
         usp = UserSettingPermission()
         usp.user = user
         usp.setting = new_s

epdb/management/commands/bootstrap.py

@@ -8,7 +8,6 @@ from epdb.logic import UserManager, GroupManager, PackageManager, SettingManager
 from epdb.models import (
     UserSettingPermission,
     MLRelativeReasoning,
-    EnviFormer,
     Permission,
     User,
     ExternalDatabase,
@@ -231,7 +230,6 @@ class Command(BaseCommand):
             package=pack,
             rule_packages=[mapping["EAWAG-BBD"]],
             data_packages=[mapping["EAWAG-BBD"]],
-            eval_packages=[],
             threshold=0.5,
             name="ECC - BBD - T0.5",
             description="ML Relative Reasoning",
@@ -239,7 +237,3 @@ class Command(BaseCommand):
 
         ml_model.build_dataset()
         ml_model.build_model()
-
-        # If available, create EnviFormerModel
-        if s.ENVIFORMER_PRESENT:
-            EnviFormer.create(pack, "EnviFormer - T0.5", "EnviFormer Model with Threshold 0.5", 0.5)

epdb/management/commands/create_api_token.py

@@ -0,0 +1,92 @@
+from django.conf import settings as s
+from django.contrib.auth import get_user_model
+from django.core.management.base import BaseCommand, CommandError
+
+from epdb.models import APIToken
+
+
+class Command(BaseCommand):
+    help = "Create an API token for a user"
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--username",
+            required=True,
+            help="Username of the user who will own the token",
+        )
+        parser.add_argument(
+            "--name",
+            required=True,
+            help="Descriptive name for the token",
+        )
+        parser.add_argument(
+            "--expires-days",
+            type=int,
+            default=90,
+            help="Days until expiration (0 for no expiration)",
+        )
+        parser.add_argument(
+            "--inactive",
+            action="store_true",
+            help="Create the token as inactive",
+        )
+        parser.add_argument(
+            "--curl",
+            action="store_true",
+            help="Print a curl example using the token",
+        )
+        parser.add_argument(
+            "--base-url",
+            default=None,
+            help="Base URL for curl example (default SERVER_URL or http://localhost:8000)",
+        )
+        parser.add_argument(
+            "--endpoint",
+            default="/api/v1/compounds/",
+            help="Endpoint path for curl example",
+        )
+
+    def handle(self, *args, **options):
+        username = options["username"]
+        name = options["name"]
+        expires_days = options["expires_days"]
+
+        if expires_days < 0:
+            raise CommandError("--expires-days must be >= 0")
+
+        if expires_days == 0:
+            expires_days = None
+
+        user_model = get_user_model()
+        try:
+            user = user_model.objects.get(username=username)
+        except user_model.DoesNotExist as exc:
+            raise CommandError(f"User not found for username '{username}'") from exc
+
+        token, raw_token = APIToken.create_token(user, name=name, expires_days=expires_days)
+
+        if options["inactive"]:
+            token.is_active = False
+            token.save(update_fields=["is_active"])
+
+        self.stdout.write(f"User: {user.username} ({user.email})")
+        self.stdout.write(f"Token name: {token.name}")
+        self.stdout.write(f"Token id: {token.id}")
+        if token.expires_at:
+            self.stdout.write(f"Expires at: {token.expires_at.isoformat()}")
+        else:
+            self.stdout.write("Expires at: never")
+        self.stdout.write(f"Active: {token.is_active}")
+        self.stdout.write("Raw token:")
+        self.stdout.write(raw_token)
+
+        if options["curl"]:
+            base_url = (
+                options["base_url"] or getattr(s, "SERVER_URL", None) or "http://localhost:8000"
+            )
+            endpoint = options["endpoint"]
+            endpoint = endpoint if endpoint.startswith("/") else f"/{endpoint}"
+            url = f"{base_url.rstrip('/')}{endpoint}"
+            curl_cmd = f'curl -H "Authorization: Bearer {raw_token}" "{url}"'
+            self.stdout.write("Curl:")
+            self.stdout.write(curl_cmd)

epdb/management/commands/dump_enviformer.py

@@ -47,7 +47,7 @@ class Command(BaseCommand):
             "description": model.description,
             "kv": model.kv,
             "data_packages_uuids": [str(p.uuid) for p in model.data_packages.all()],
-            "eval_packages_uuids": [str(p.uuid) for p in model.data_packages.all()],
+            "eval_packages_uuids": [str(p.uuid) for p in model.eval_packages.all()],
             "threshold": model.threshold,
             "eval_results": model.eval_results,
             "multigen_eval": model.multigen_eval,

epdb/management/commands/localize_urls.py

@@ -41,9 +41,7 @@ class Command(BaseCommand):
             "SequentialRule",
             "Scenario",
             "Setting",
-            "MLRelativeReasoning",
-            "RuleBasedRelativeReasoning",
-            "EnviFormer",
+            "EPModel",
             "ApplicabilityDomain",
             "EnzymeLink",
         ]

epdb/management/commands/recreate_db.py

@@ -0,0 +1,83 @@
+import os
+import subprocess
+
+from django.conf import settings
+from django.core.management import call_command
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "-n",
+            "--name",
+            type=str,
+            help="Name of the database to recreate. Default is 'appdb'",
+            default="appdb",
+        )
+
+        parser.add_argument(
+            "-d",
+            "--dump",
+            type=str,
+            help="Path to the dump file",
+            default="./fixtures/db.dump",
+        )
+
+        parser.add_argument(
+            "-ou",
+            "--oldurl",
+            type=str,
+            help="Old URL, e.g. https://envipath.org/",
+            default="https://envipath.org/",
+        )
+
+        parser.add_argument(
+            "-nu",
+            "--newurl",
+            type=str,
+            help="New URL, e.g. http://localhost:8000/",
+            default="http://localhost:8000/",
+        )
+
+    def handle(self, *args, **options):
+        dump_file = options["dump"]
+
+        if not os.path.exists(dump_file):
+            raise ValueError(f"Dump file {dump_file} does not exist")
+
+        db_name = options["name"]
+
+        print(f"Dropping database {db_name} y/n: ", end="")
+
+        if input() in "yY":
+            result = subprocess.run(
+                ["dropdb", db_name],
+                capture_output=True,
+                text=True,
+            )
+            print(result.stdout)
+        else:
+            raise ValueError("Aborted")
+
+        print(f"Creating database {db_name}")
+
+        result = subprocess.run(
+            ["createdb", db_name],
+            capture_output=True,
+            text=True,
+        )
+        print(result.stdout)
+        print(f"Restoring database {db_name} from {dump_file}")
+
+        result = subprocess.run(
+            ["pg_restore", "-d", db_name, dump_file, "--no-owner"],
+            capture_output=True,
+            text=True,
+        )
+        print(result.stdout)
+
+        if db_name == settings.DATABASES["default"]["NAME"]:
+            call_command("localize_urls", "--old", options["oldurl"], "--new", options["newurl"])
+        else:
+            print("Skipping localize_urls as database is not the default one.")

epdb/migrations/0002_externaldatabase_alter_apitoken_options_and_more.py

@@ -1,128 +0,0 @@
-# Generated by Django 5.2.1 on 2025-08-25 18:07
-
-import django.db.models.deletion
-import django.utils.timezone
-import model_utils.fields
-import uuid
-from django.conf import settings
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('contenttypes', '0002_remove_content_type_name'),
-        ('epdb', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.CreateModel(
-            name='ExternalDatabase',
-            fields=[
-                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
-                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
-                ('uuid', models.UUIDField(default=uuid.uuid4, editable=False, unique=True)),
-                ('name', models.CharField(max_length=100, unique=True, verbose_name='Database Name')),
-                ('full_name', models.CharField(blank=True, max_length=255, verbose_name='Full Database Name')),
-                ('description', models.TextField(blank=True, verbose_name='Description')),
-                ('base_url', models.URLField(blank=True, null=True, verbose_name='Base URL')),
-                ('url_pattern', models.CharField(blank=True, help_text="URL pattern with {id} placeholder, e.g., 'https://pubchem.ncbi.nlm.nih.gov/compound/{id}'", max_length=500, verbose_name='URL Pattern')),
-                ('is_active', models.BooleanField(default=True, verbose_name='Is Active')),
-            ],
-            options={
-                'verbose_name': 'External Database',
-                'verbose_name_plural': 'External Databases',
-                'db_table': 'epdb_external_database',
-                'ordering': ['name'],
-            },
-        ),
-        migrations.AlterModelOptions(
-            name='apitoken',
-            options={'ordering': ['-created'], 'verbose_name': 'API Token', 'verbose_name_plural': 'API Tokens'},
-        ),
-        migrations.AlterModelOptions(
-            name='edge',
-            options={},
-        ),
-        migrations.RemoveField(
-            model_name='edge',
-            name='polymorphic_ctype',
-        ),
-        migrations.AddField(
-            model_name='apitoken',
-            name='is_active',
-            field=models.BooleanField(default=True, help_text='Whether this token is active'),
-        ),
-        migrations.AddField(
-            model_name='apitoken',
-            name='modified',
-            field=model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified'),
-        ),
-        migrations.AddField(
-            model_name='applicabilitydomain',
-            name='functional_groups',
-            field=models.JSONField(blank=True, default=dict, null=True),
-        ),
-        migrations.AddField(
-            model_name='mlrelativereasoning',
-            name='app_domain',
-            field=models.ForeignKey(blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.applicabilitydomain'),
-        ),
-        migrations.AlterField(
-            model_name='apitoken',
-            name='created',
-            field=model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created'),
-        ),
-        migrations.AlterField(
-            model_name='apitoken',
-            name='expires_at',
-            field=models.DateTimeField(blank=True, help_text='Token expiration time (null for no expiration)', null=True),
-        ),
-        migrations.AlterField(
-            model_name='apitoken',
-            name='hashed_key',
-            field=models.CharField(help_text='SHA-256 hash of the token key', max_length=128, unique=True),
-        ),
-        migrations.AlterField(
-            model_name='apitoken',
-            name='name',
-            field=models.CharField(help_text='Descriptive name for this token', max_length=100),
-        ),
-        migrations.AlterField(
-            model_name='apitoken',
-            name='user',
-            field=models.ForeignKey(help_text='User who owns this token', on_delete=django.db.models.deletion.CASCADE, related_name='api_tokens', to=settings.AUTH_USER_MODEL),
-        ),
-        migrations.AlterField(
-            model_name='applicabilitydomain',
-            name='num_neighbours',
-            field=models.IntegerField(default=5),
-        ),
-        migrations.AlterModelTable(
-            name='apitoken',
-            table='epdb_api_token',
-        ),
-        migrations.CreateModel(
-            name='ExternalIdentifier',
-            fields=[
-                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
-                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
-                ('uuid', models.UUIDField(default=uuid.uuid4, editable=False, unique=True)),
-                ('object_id', models.IntegerField()),
-                ('identifier_value', models.CharField(max_length=255, verbose_name='Identifier Value')),
-                ('url', models.URLField(blank=True, null=True, verbose_name='Direct URL')),
-                ('is_primary', models.BooleanField(default=False, help_text='Mark this as the primary identifier for this database', verbose_name='Is Primary')),
-                ('content_type', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='contenttypes.contenttype')),
-                ('database', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.externaldatabase', verbose_name='External Database')),
-            ],
-            options={
-                'verbose_name': 'External Identifier',
-                'verbose_name_plural': 'External Identifiers',
-                'db_table': 'epdb_external_identifier',
-                'indexes': [models.Index(fields=['content_type', 'object_id'], name='epdb_extern_content_b76813_idx'), models.Index(fields=['database', 'identifier_value'], name='epdb_extern_databas_486422_idx')],
-                'unique_together': {('content_type', 'object_id', 'database', 'identifier_value')},
-            },
-        ),
-    ]

epdb/migrations/0003_applicabilitydomain_url_compound_url_and_more.py

@@ -1,228 +0,0 @@
-# Generated by Django 5.2.1 on 2025-08-26 17:05
-
-from django.db import migrations, models
-
-
-def populate_url(apps, schema_editor):
-    MODELS = [
-        'User',
-        'Group',
-        'Package',
-        'Compound',
-        'CompoundStructure',
-        'Pathway',
-        'Edge',
-        'Node',
-        'Reaction',
-        'SimpleAmbitRule',
-        'SimpleRDKitRule',
-        'ParallelRule',
-        'SequentialRule',
-        'Scenario',
-        'Setting',
-        'MLRelativeReasoning',
-        'EnviFormer',
-        'ApplicabilityDomain',
-    ]
-    for model in MODELS:
-        obj_cls = apps.get_model("epdb", model)
-        for obj in obj_cls.objects.all():
-            obj.url = assemble_url(obj)
-            if obj.url is None:
-                raise ValueError(f"Could not assemble url for {obj}")
-            obj.save()
-
-
-def assemble_url(obj):
-    from django.conf import settings as s
-    match obj.__class__.__name__:
-        case 'User':
-            return '{}/user/{}'.format(s.SERVER_URL, obj.uuid)
-        case 'Group':
-            return '{}/group/{}'.format(s.SERVER_URL, obj.uuid)
-        case 'Package':
-            return '{}/package/{}'.format(s.SERVER_URL, obj.uuid)
-        case 'Compound':
-            return '{}/compound/{}'.format(obj.package.url, obj.uuid)
-        case 'CompoundStructure':
-            return '{}/structure/{}'.format(obj.compound.url, obj.uuid)
-        case 'SimpleAmbitRule':
-            return '{}/simple-ambit-rule/{}'.format(obj.package.url, obj.uuid)
-        case 'SimpleRDKitRule':
-            return '{}/simple-rdkit-rule/{}'.format(obj.package.url, obj.uuid)
-        case 'ParallelRule':
-            return '{}/parallel-rule/{}'.format(obj.package.url, obj.uuid)
-        case 'SequentialRule':
-            return '{}/sequential-rule/{}'.format(obj.compound.url, obj.uuid)
-        case 'Reaction':
-            return '{}/reaction/{}'.format(obj.package.url, obj.uuid)
-        case 'Pathway':
-            return '{}/pathway/{}'.format(obj.package.url, obj.uuid)
-        case 'Node':
-            return '{}/node/{}'.format(obj.pathway.url, obj.uuid)
-        case 'Edge':
-            return '{}/edge/{}'.format(obj.pathway.url, obj.uuid)
-        case 'MLRelativeReasoning':
-            return '{}/model/{}'.format(obj.package.url, obj.uuid)
-        case 'EnviFormer':
-            return '{}/model/{}'.format(obj.package.url, obj.uuid)
-        case 'ApplicabilityDomain':
-            return '{}/model/{}/applicability-domain/{}'.format(obj.model.package.url, obj.model.uuid, obj.uuid)
-        case 'Scenario':
-            return '{}/scenario/{}'.format(obj.package.url, obj.uuid)
-        case 'Setting':
-            return '{}/setting/{}'.format(s.SERVER_URL, obj.uuid)
-        case _:
-            raise ValueError(f"Unknown model {obj.__class__.__name__}")
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ('epdb', '0002_externaldatabase_alter_apitoken_options_and_more'),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name='applicabilitydomain',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='compound',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='compoundstructure',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='edge',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='epmodel',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='group',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='node',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='package',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='pathway',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='reaction',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='rule',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='scenario',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='setting',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-        migrations.AddField(
-            model_name='user',
-            name='url',
-            field=models.TextField(null=True, unique=False, verbose_name='URL'),
-        ),
-
-        migrations.RunPython(populate_url, reverse_code=migrations.RunPython.noop),
-
-        migrations.AlterField(
-            model_name='applicabilitydomain',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='compound',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='compoundstructure',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='edge',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='epmodel',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='group',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='node',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='package',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='pathway',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='reaction',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='rule',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='scenario',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='setting',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-        migrations.AlterField(
-            model_name='user',
-            name='url',
-            field=models.TextField(null=True, unique=True, verbose_name='URL'),
-        ),
-    ]

epdb/migrations/0004_alter_mlrelativereasoning_options_and_more.py

@@ -1,55 +0,0 @@
-# Generated by Django 5.2.1 on 2025-09-09 09:21
-
-import django.db.models.deletion
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('epdb', '0001_squashed_0003_applicabilitydomain_url_compound_url_and_more'),
-    ]
-
-    operations = [
-        migrations.AlterModelOptions(
-            name='mlrelativereasoning',
-            options={},
-        ),
-        migrations.AlterField(
-            model_name='mlrelativereasoning',
-            name='data_packages',
-            field=models.ManyToManyField(related_name='%(app_label)s_%(class)s_data_packages', to='epdb.package', verbose_name='Data Packages'),
-        ),
-        migrations.AlterField(
-            model_name='mlrelativereasoning',
-            name='eval_packages',
-            field=models.ManyToManyField(related_name='%(app_label)s_%(class)s_eval_packages', to='epdb.package', verbose_name='Evaluation Packages'),
-        ),
-        migrations.AlterField(
-            model_name='mlrelativereasoning',
-            name='rule_packages',
-            field=models.ManyToManyField(related_name='%(app_label)s_%(class)s_rule_packages', to='epdb.package', verbose_name='Rule Packages'),
-        ),
-        migrations.CreateModel(
-            name='RuleBasedRelativeReasoning',
-            fields=[
-                ('epmodel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='epdb.epmodel')),
-                ('threshold', models.FloatField(default=0.5)),
-                ('eval_results', models.JSONField(blank=True, default=dict, null=True)),
-                ('model_status', models.CharField(choices=[('INITIAL', 'Initial'), ('INITIALIZING', 'Model is initializing.'), ('BUILDING', 'Model is building.'), ('BUILT_NOT_EVALUATED', 'Model is built and can be used for predictions, Model is not evaluated yet.'), ('EVALUATING', 'Model is evaluating'), ('FINISHED', 'Model has finished building and evaluation.'), ('ERROR', 'Model has failed.')], default='INITIAL')),
-                ('min_count', models.IntegerField(default=10)),
-                ('max_count', models.IntegerField(default=0)),
-                ('app_domain', models.ForeignKey(blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.applicabilitydomain')),
-                ('data_packages', models.ManyToManyField(related_name='%(app_label)s_%(class)s_data_packages', to='epdb.package', verbose_name='Data Packages')),
-                ('eval_packages', models.ManyToManyField(related_name='%(app_label)s_%(class)s_eval_packages', to='epdb.package', verbose_name='Evaluation Packages')),
-                ('rule_packages', models.ManyToManyField(related_name='%(app_label)s_%(class)s_rule_packages', to='epdb.package', verbose_name='Rule Packages')),
-            ],
-            options={
-                'abstract': False,
-            },
-            bases=('epdb.epmodel',),
-        ),
-        migrations.DeleteModel(
-            name='RuleBaseRelativeReasoning',
-        ),
-    ]

epdb/migrations/0005_alter_group_group_member.py

@@ -1,18 +0,0 @@
-# Generated by Django 5.2.1 on 2025-09-11 06:21
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('epdb', '0004_alter_mlrelativereasoning_options_and_more'),
-    ]
-
-    operations = [
-        migrations.AlterField(
-            model_name='group',
-            name='group_member',
-            field=models.ManyToManyField(blank=True, related_name='groups_in_group', to='epdb.group', verbose_name='Group member'),
-        ),
-    ]

epdb/migrations/0006_mlrelativereasoning_multigen_eval_and_more.py

@@ -1,23 +0,0 @@
-# Generated by Django 5.2.1 on 2025-09-18 06:42
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('epdb', '0005_alter_group_group_member'),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name='mlrelativereasoning',
-            name='multigen_eval',
-            field=models.BooleanField(default=False),
-        ),
-        migrations.AddField(
-            model_name='rulebasedrelativereasoning',
-            name='multigen_eval',
-            field=models.BooleanField(default=False),
-        ),
-    ]

epdb/migrations/0007_alter_enviformer_options_enviformer_app_domain_and_more.py

@@ -1,53 +0,0 @@
-# Generated by Django 5.2.1 on 2025-10-07 08:19
-
-import django.db.models.deletion
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('epdb', '0006_mlrelativereasoning_multigen_eval_and_more'),
-    ]
-
-    operations = [
-        migrations.AlterModelOptions(
-            name='enviformer',
-            options={},
-        ),
-        migrations.AddField(
-            model_name='enviformer',
-            name='app_domain',
-            field=models.ForeignKey(blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.applicabilitydomain'),
-        ),
-        migrations.AddField(
-            model_name='enviformer',
-            name='data_packages',
-            field=models.ManyToManyField(related_name='%(app_label)s_%(class)s_data_packages', to='epdb.package', verbose_name='Data Packages'),
-        ),
-        migrations.AddField(
-            model_name='enviformer',
-            name='eval_packages',
-            field=models.ManyToManyField(related_name='%(app_label)s_%(class)s_eval_packages', to='epdb.package', verbose_name='Evaluation Packages'),
-        ),
-        migrations.AddField(
-            model_name='enviformer',
-            name='eval_results',
-            field=models.JSONField(blank=True, default=dict, null=True),
-        ),
-        migrations.AddField(
-            model_name='enviformer',
-            name='model_status',
-            field=models.CharField(choices=[('INITIAL', 'Initial'), ('INITIALIZING', 'Model is initializing.'), ('BUILDING', 'Model is building.'), ('BUILT_NOT_EVALUATED', 'Model is built and can be used for predictions, Model is not evaluated yet.'), ('EVALUATING', 'Model is evaluating'), ('FINISHED', 'Model has finished building and evaluation.'), ('ERROR', 'Model has failed.')], default='INITIAL'),
-        ),
-        migrations.AddField(
-            model_name='enviformer',
-            name='multigen_eval',
-            field=models.BooleanField(default=False),
-        ),
-        migrations.AddField(
-            model_name='enviformer',
-            name='rule_packages',
-            field=models.ManyToManyField(related_name='%(app_label)s_%(class)s_rule_packages', to='epdb.package', verbose_name='Rule Packages'),
-        ),
-    ]

epdb/migrations/0008_enzymelink.py

@@ -1,64 +0,0 @@
-# Generated by Django 5.2.7 on 2025-10-10 06:58
-
-import django.db.models.deletion
-import django.utils.timezone
-import model_utils.fields
-import uuid
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("epdb", "0007_alter_enviformer_options_enviformer_app_domain_and_more"),
-    ]
-
-    operations = [
-        migrations.CreateModel(
-            name="EnzymeLink",
-            fields=[
-                (
-                    "id",
-                    models.BigAutoField(
-                        auto_created=True, primary_key=True, serialize=False, verbose_name="ID"
-                    ),
-                ),
-                (
-                    "created",
-                    model_utils.fields.AutoCreatedField(
-                        default=django.utils.timezone.now, editable=False, verbose_name="created"
-                    ),
-                ),
-                (
-                    "modified",
-                    model_utils.fields.AutoLastModifiedField(
-                        default=django.utils.timezone.now, editable=False, verbose_name="modified"
-                    ),
-                ),
-                (
-                    "uuid",
-                    models.UUIDField(
-                        default=uuid.uuid4, unique=True, verbose_name="UUID of this object"
-                    ),
-                ),
-                ("name", models.TextField(default="no name", verbose_name="Name")),
-                (
-                    "description",
-                    models.TextField(default="no description", verbose_name="Descriptions"),
-                ),
-                ("url", models.TextField(null=True, unique=True, verbose_name="URL")),
-                ("kv", models.JSONField(blank=True, default=dict, null=True)),
-                ("ec_number", models.TextField(verbose_name="EC Number")),
-                ("classification_level", models.IntegerField(verbose_name="Classification Level")),
-                ("linking_method", models.TextField(verbose_name="Linking Method")),
-                ("edge_evidence", models.ManyToManyField(to="epdb.edge")),
-                ("reaction_evidence", models.ManyToManyField(to="epdb.reaction")),
-                (
-                    "rule",
-                    models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to="epdb.rule"),
-                ),
-            ],
-            options={
-                "abstract": False,
-            },
-        ),
-    ]

epdb/migrations/0009_joblog.py

@@ -1,66 +0,0 @@
-# Generated by Django 5.2.7 on 2025-10-27 09:39
-
-import django.db.models.deletion
-import django.utils.timezone
-import model_utils.fields
-from django.conf import settings
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("epdb", "0008_enzymelink"),
-    ]
-
-    operations = [
-        migrations.CreateModel(
-            name="JobLog",
-            fields=[
-                (
-                    "id",
-                    models.BigAutoField(
-                        auto_created=True, primary_key=True, serialize=False, verbose_name="ID"
-                    ),
-                ),
-                (
-                    "created",
-                    model_utils.fields.AutoCreatedField(
-                        default=django.utils.timezone.now, editable=False, verbose_name="created"
-                    ),
-                ),
-                (
-                    "modified",
-                    model_utils.fields.AutoLastModifiedField(
-                        default=django.utils.timezone.now, editable=False, verbose_name="modified"
-                    ),
-                ),
-                ("task_id", models.UUIDField(unique=True)),
-                ("job_name", models.TextField()),
-                (
-                    "status",
-                    models.CharField(
-                        choices=[
-                            ("INITIAL", "Initial"),
-                            ("SUCCESS", "Success"),
-                            ("FAILURE", "Failure"),
-                            ("REVOKED", "Revoked"),
-                            ("IGNORED", "Ignored"),
-                        ],
-                        default="INITIAL",
-                        max_length=20,
-                    ),
-                ),
-                ("done_at", models.DateTimeField(blank=True, default=None, null=True)),
-                ("task_result", models.TextField(blank=True, default=None, null=True)),
-                (
-                    "user",
-                    models.ForeignKey(
-                        on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL
-                    ),
-                ),
-            ],
-            options={
-                "abstract": False,
-            },
-        ),
-    ]

epdb/migrations/0010_license_cc_string.py

@@ -1,18 +0,0 @@
-# Generated by Django 5.2.7 on 2025-11-11 14:11
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("epdb", "0009_joblog"),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name="license",
-            name="cc_string",
-            field=models.TextField(default="by-nc-sa", verbose_name="CC string"),
-            preserve_default=False,
-        ),
-    ]

epdb/migrations/0011_auto_20251111_1413.py

@@ -1,59 +0,0 @@
-# Generated by Django 5.2.7 on 2025-11-11 14:13
-
-import re
-
-from django.contrib.postgres.aggregates import ArrayAgg
-from django.db import migrations
-from django.db.models import Min
-
-
-def set_cc(apps, schema_editor):
-    License = apps.get_model("epdb", "License")
-
-    # For all existing licenses extract cc_string from link
-    for license in License.objects.all():
-        pattern = r"/licenses/([^/]+)/4\.0"
-        match = re.search(pattern, license.link)
-        if match:
-            license.cc_string = match.group(1)
-            license.save()
-        else:
-            raise ValueError(f"Could not find license for {license.link}")
-
-    # Ensure we have all licenses
-    cc_strings = ["by", "by-nc", "by-nc-nd", "by-nc-sa", "by-nd", "by-sa"]
-    for cc_string in cc_strings:
-        if not License.objects.filter(cc_string=cc_string).exists():
-            new_license = License()
-            new_license.cc_string = cc_string
-            new_license.link = f"https://creativecommons.org/licenses/{cc_string}/4.0/"
-            new_license.image_link = f"https://licensebuttons.net/l/{cc_string}/4.0/88x31.png"
-            new_license.save()
-
-    # As we might have existing Licenses representing the same License,
-    # get min pk and all pks as a list
-    license_lookup_qs = License.objects.values("cc_string").annotate(
-        lowest_pk=Min("id"), all_pks=ArrayAgg("id", order_by=("id",))
-    )
-
-    license_lookup = {
-        row["cc_string"]: (row["lowest_pk"], row["all_pks"]) for row in license_lookup_qs
-    }
-
-    Packages = apps.get_model("epdb", "Package")
-
-    for k, v in license_lookup.items():
-        # Set min pk to all packages pointing to any of the duplicates
-        Packages.objects.filter(pk__in=v[1]).update(license_id=v[0])
-        # remove the min pk from "other" pks as we use them for deletion
-        v[1].remove(v[0])
-        # Delete redundant License objects
-        License.objects.filter(pk__in=v[1]).delete()
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("epdb", "0010_license_cc_string"),
-    ]
-
-    operations = [migrations.RunPython(set_cc)]

epdb/migrations/0012_node_stereo_removed_pathway_predicted.py

@@ -1,22 +0,0 @@
-# Generated by Django 5.2.7 on 2025-12-02 13:09
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("epdb", "0011_auto_20251111_1413"),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name="node",
-            name="stereo_removed",
-            field=models.BooleanField(default=False),
-        ),
-        migrations.AddField(
-            model_name="pathway",
-            name="predicted",
-            field=models.BooleanField(default=False),
-        ),
-    ]

epdb/migrations/0013_setting_expansion_schema.py

@@ -1,25 +0,0 @@
-# Generated by Django 5.2.7 on 2025-12-14 11:30
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("epdb", "0012_node_stereo_removed_pathway_predicted"),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name="setting",
-            name="expansion_schema",
-            field=models.CharField(
-                choices=[
-                    ("BFS", "Breadth First Search"),
-                    ("DFS", "Depth First Search"),
-                    ("GREEDY", "Greedy"),
-                ],
-                default="BFS",
-                max_length=20,
-            ),
-        ),
-    ]

epdb/migrations/0014_rename_expansion_schema_setting_expansion_scheme.py

@@ -1,17 +0,0 @@
-# Generated by Django 5.2.7 on 2025-12-14 16:02
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("epdb", "0013_setting_expansion_schema"),
-    ]
-
-    operations = [
-        migrations.RenameField(
-            model_name="setting",
-            old_name="expansion_schema",
-            new_name="expansion_scheme",
-        ),
-    ]

epdb/migrations/0019_remove_scenario_additional_information_and_more.py

@@ -1,6 +1,5 @@
-# Generated by Django 5.2.1 on 2025-07-22 20:58
+# Generated by Django 5.2.7 on 2026-03-06 10:51
 
-import datetime
 import django.contrib.auth.models
 import django.contrib.auth.validators
 import django.contrib.postgres.fields
@@ -19,11 +18,12 @@ class Migration(migrations.Migration):
     dependencies = [
         ('auth', '0012_alter_user_first_name_max_length'),
         ('contenttypes', '0002_remove_content_type_name'),
+        migrations.swappable_dependency(settings.EPDB_PACKAGE_MODEL),
     ]
 
     operations = [
         migrations.CreateModel(
-            name='Compound',
+            name='ApplicabilityDomain',
             fields=[
                 ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                 ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
@@ -31,9 +31,33 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
+                ('kv', models.JSONField(blank=True, default=dict, null=True)),
+                ('num_neighbours', models.IntegerField(default=5)),
+                ('reliability_threshold', models.FloatField(default=0.5)),
+                ('local_compatibilty_threshold', models.FloatField(default=0.5)),
+                ('functional_groups', models.JSONField(blank=True, default=dict, null=True)),
+            ],
+            options={
+                'abstract': False,
+            },
+        ),
+        migrations.CreateModel(
+            name='Edge',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
+                ('name', models.TextField(default='no name', verbose_name='Name')),
+                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
             ],
+            options={
+                'abstract': False,
+            },
         ),
         migrations.CreateModel(
             name='EPModel',
@@ -44,7 +68,10 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
+                ('model_status', models.CharField(choices=[('INITIAL', 'Initial'), ('INITIALIZING', 'Model is initializing.'), ('BUILDING', 'Model is building.'), ('BUILT_NOT_EVALUATED', 'Model is built and can be used for predictions, Model is not evaluated yet.'), ('EVALUATING', 'Model is evaluating'), ('FINISHED', 'Model has finished building and evaluation.'), ('ERROR', 'Model has failed.')], default='INITIAL')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Package')),
                 ('polymorphic_ctype', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='polymorphic_%(app_label)s.%(class)s_set+', to='contenttypes.contenttype')),
             ],
             options={
@@ -52,6 +79,27 @@ class Migration(migrations.Migration):
                 'base_manager_name': 'objects',
             },
         ),
+        migrations.CreateModel(
+            name='ExternalDatabase',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, editable=False, unique=True)),
+                ('name', models.CharField(max_length=100, unique=True, verbose_name='Database Name')),
+                ('full_name', models.CharField(blank=True, max_length=255, verbose_name='Full Database Name')),
+                ('description', models.TextField(blank=True, verbose_name='Description')),
+                ('base_url', models.URLField(blank=True, null=True, verbose_name='Base URL')),
+                ('url_pattern', models.CharField(blank=True, help_text="URL pattern with {id} placeholder, e.g., 'https://pubchem.ncbi.nlm.nih.gov/compound/{id}'", max_length=500, verbose_name='URL Pattern')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Is Active')),
+            ],
+            options={
+                'verbose_name': 'External Database',
+                'verbose_name_plural': 'External Databases',
+                'db_table': 'epdb_external_database',
+                'ordering': ['name'],
+            },
+        ),
         migrations.CreateModel(
             name='Permission',
             fields=[
@@ -65,6 +113,7 @@ class Migration(migrations.Migration):
             name='License',
             fields=[
                 ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('cc_string', models.TextField(verbose_name='CC string')),
                 ('link', models.URLField(verbose_name='link')),
                 ('image_link', models.URLField(verbose_name='Image link')),
             ],
@@ -78,8 +127,11 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Package')),
+                ('polymorphic_ctype', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='polymorphic_%(app_label)s.%(class)s_set+', to='contenttypes.contenttype')),
             ],
             options={
                 'abstract': False,
@@ -101,6 +153,9 @@ class Migration(migrations.Migration):
                 ('date_joined', models.DateTimeField(default=django.utils.timezone.now, verbose_name='date joined')),
                 ('email', models.EmailField(max_length=254, unique=True)),
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
+                ('is_reviewer', models.BooleanField(default=False)),
+                ('default_package', models.ForeignKey(null=True, on_delete=django.db.models.deletion.SET_NULL, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Default Package')),
                 ('groups', models.ManyToManyField(blank=True, help_text='The groups this user belongs to. A user will get all permissions granted to each of their groups.', related_name='user_set', related_query_name='user', to='auth.group', verbose_name='groups')),
                 ('user_permissions', models.ManyToManyField(blank=True, help_text='Specific permissions for this user.', related_name='user_set', related_query_name='user', to='auth.permission', verbose_name='user permissions')),
             ],
@@ -117,11 +172,34 @@ class Migration(migrations.Migration):
             name='APIToken',
             fields=[
                 ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('hashed_key', models.CharField(max_length=128, unique=True)),
-                ('created', models.DateTimeField(auto_now_add=True)),
-                ('expires_at', models.DateTimeField(blank=True, default=datetime.datetime(2025, 10, 20, 20, 58, 48, 351675, tzinfo=datetime.timezone.utc), null=True)),
-                ('name', models.CharField(blank=True, help_text='Optional name for the token', max_length=100)),
-                ('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('hashed_key', models.CharField(help_text='SHA-256 hash of the token key', max_length=128, unique=True)),
+                ('expires_at', models.DateTimeField(blank=True, help_text='Token expiration time (null for no expiration)', null=True)),
+                ('name', models.CharField(help_text='Descriptive name for this token', max_length=100)),
+                ('is_active', models.BooleanField(default=True, help_text='Whether this token is active')),
+                ('user', models.ForeignKey(help_text='User who owns this token', on_delete=django.db.models.deletion.CASCADE, related_name='api_tokens', to=settings.AUTH_USER_MODEL)),
+            ],
+            options={
+                'verbose_name': 'API Token',
+                'verbose_name_plural': 'API Tokens',
+                'db_table': 'epdb_api_token',
+                'ordering': ['-created'],
+            },
+        ),
+        migrations.CreateModel(
+            name='Compound',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
+                ('name', models.TextField(default='no name', verbose_name='Name')),
+                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
+                ('kv', models.JSONField(blank=True, default=dict, null=True)),
+                ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Package')),
             ],
         ),
         migrations.CreateModel(
@@ -133,6 +211,7 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
                 ('smiles', models.TextField(verbose_name='SMILES')),
@@ -151,54 +230,16 @@ class Migration(migrations.Migration):
             field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, related_name='compound_default_structure', to='epdb.compoundstructure', verbose_name='Default Structure'),
         ),
         migrations.CreateModel(
-            name='Edge',
-            fields=[
-                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
-                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
-                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
-                ('name', models.TextField(default='no name', verbose_name='Name')),
-                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
-                ('kv', models.JSONField(blank=True, default=dict, null=True)),
-                ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
-                ('polymorphic_ctype', models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='polymorphic_%(app_label)s.%(class)s_set+', to='contenttypes.contenttype')),
-            ],
-            options={
-                'abstract': False,
-                'base_manager_name': 'objects',
-            },
-        ),
-        migrations.CreateModel(
-            name='EnviFormer',
+            name='PropertyPluginModel',
             fields=[
                 ('epmodel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='epdb.epmodel')),
                 ('threshold', models.FloatField(default=0.5)),
+                ('eval_results', models.JSONField(blank=True, default=dict, null=True)),
+                ('multigen_eval', models.BooleanField(default=False)),
+                ('plugin_identifier', models.CharField(max_length=255)),
             ],
             options={
                 'abstract': False,
-                'base_manager_name': 'objects',
-            },
-            bases=('epdb.epmodel',),
-        ),
-        migrations.CreateModel(
-            name='PluginModel',
-            fields=[
-                ('epmodel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='epdb.epmodel')),
-            ],
-            options={
-                'abstract': False,
-                'base_manager_name': 'objects',
-            },
-            bases=('epdb.epmodel',),
-        ),
-        migrations.CreateModel(
-            name='RuleBaseRelativeReasoning',
-            fields=[
-                ('epmodel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='epdb.epmodel')),
-            ],
-            options={
-                'abstract': False,
-                'base_manager_name': 'objects',
             },
             bases=('epdb.epmodel',),
         ),
@@ -209,10 +250,11 @@ class Migration(migrations.Migration):
                 ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
                 ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('name', models.TextField(verbose_name='Group name')),
                 ('public', models.BooleanField(default=False, verbose_name='Public Group')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
-                ('group_member', models.ManyToManyField(related_name='groups_in_group', to='epdb.group', verbose_name='Group member')),
+                ('group_member', models.ManyToManyField(blank=True, related_name='groups_in_group', to='epdb.group', verbose_name='Group member')),
                 ('owner', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL, verbose_name='Group Owner')),
                 ('user_member', models.ManyToManyField(related_name='users_in_group', to=settings.AUTH_USER_MODEL, verbose_name='User members')),
             ],
@@ -225,6 +267,41 @@ class Migration(migrations.Migration):
             name='default_group',
             field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='default_group', to='epdb.group', verbose_name='Default Group'),
         ),
+        migrations.CreateModel(
+            name='JobLog',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('task_id', models.UUIDField(unique=True)),
+                ('job_name', models.TextField()),
+                ('status', models.CharField(choices=[('INITIAL', 'Initial'), ('SUCCESS', 'Success'), ('FAILURE', 'Failure'), ('REVOKED', 'Revoked'), ('IGNORED', 'Ignored')], default='INITIAL', max_length=20)),
+                ('done_at', models.DateTimeField(blank=True, default=None, null=True)),
+                ('task_result', models.TextField(blank=True, default=None, null=True)),
+                ('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
+            ],
+            options={
+                'abstract': False,
+            },
+        ),
+        migrations.CreateModel(
+            name='Package',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
+                ('name', models.TextField(default='no name', verbose_name='Name')),
+                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
+                ('kv', models.JSONField(blank=True, default=dict, null=True)),
+                ('reviewed', models.BooleanField(default=False, verbose_name='Reviewstatus')),
+                ('license', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.license', verbose_name='License')),
+            ],
+            options={
+                'swappable': 'EPDB_PACKAGE_MODEL',
+            },
+        ),
         migrations.CreateModel(
             name='Node',
             fields=[
@@ -234,9 +311,11 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
                 ('depth', models.IntegerField(verbose_name='Node depth')),
+                ('stereo_removed', models.BooleanField(default=False)),
                 ('default_node_label', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='default_node_structure', to='epdb.compoundstructure', verbose_name='Default Node Label')),
                 ('node_labels', models.ManyToManyField(related_name='node_structures', to='epdb.compoundstructure', verbose_name='All Node Labels')),
                 ('out_edges', models.ManyToManyField(to='epdb.edge', verbose_name='Outgoing Edges')),
@@ -255,38 +334,6 @@ class Migration(migrations.Migration):
             name='start_nodes',
             field=models.ManyToManyField(related_name='edge_educts', to='epdb.node', verbose_name='Start Nodes'),
         ),
-        migrations.CreateModel(
-            name='Package',
-            fields=[
-                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
-                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
-                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
-                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
-                ('name', models.TextField(default='no name', verbose_name='Name')),
-                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
-                ('kv', models.JSONField(blank=True, default=dict, null=True)),
-                ('reviewed', models.BooleanField(default=False, verbose_name='Reviewstatus')),
-                ('license', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.license', verbose_name='License')),
-            ],
-            options={
-                'abstract': False,
-            },
-        ),
-        migrations.AddField(
-            model_name='epmodel',
-            name='package',
-            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Package'),
-        ),
-        migrations.AddField(
-            model_name='compound',
-            name='package',
-            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Package'),
-        ),
-        migrations.AddField(
-            model_name='user',
-            name='default_package',
-            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.package', verbose_name='Default Package'),
-        ),
         migrations.CreateModel(
             name='SequentialRule',
             fields=[
@@ -309,16 +356,6 @@ class Migration(migrations.Migration):
             },
             bases=('epdb.rule',),
         ),
-        migrations.AddField(
-            model_name='rule',
-            name='package',
-            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Package'),
-        ),
-        migrations.AddField(
-            model_name='rule',
-            name='polymorphic_ctype',
-            field=models.ForeignKey(editable=False, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='polymorphic_%(app_label)s.%(class)s_set+', to='contenttypes.contenttype'),
-        ),
         migrations.CreateModel(
             name='Pathway',
             fields=[
@@ -328,9 +365,11 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
-                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Package')),
+                ('predicted', models.BooleanField(default=False)),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Package')),
             ],
             options={
                 'abstract': False,
@@ -355,12 +394,13 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('aliases', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), default=list, size=None, verbose_name='Aliases')),
                 ('multi_step', models.BooleanField(verbose_name='Multistep Reaction')),
                 ('medline_references', django.contrib.postgres.fields.ArrayField(base_field=models.TextField(), null=True, size=None, verbose_name='Medline References')),
                 ('educts', models.ManyToManyField(related_name='reaction_educts', to='epdb.compoundstructure', verbose_name='Educts')),
-                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Package')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Package')),
                 ('products', models.ManyToManyField(related_name='reaction_products', to='epdb.compoundstructure', verbose_name='Products')),
                 ('rules', models.ManyToManyField(related_name='reaction_rule', to='epdb.rule', verbose_name='Rule')),
             ],
@@ -368,6 +408,28 @@ class Migration(migrations.Migration):
                 'abstract': False,
             },
         ),
+        migrations.CreateModel(
+            name='EnzymeLink',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
+                ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
+                ('name', models.TextField(default='no name', verbose_name='Name')),
+                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
+                ('kv', models.JSONField(blank=True, default=dict, null=True)),
+                ('ec_number', models.TextField(verbose_name='EC Number')),
+                ('classification_level', models.IntegerField(verbose_name='Classification Level')),
+                ('linking_method', models.TextField(verbose_name='Linking Method')),
+                ('edge_evidence', models.ManyToManyField(to='epdb.edge')),
+                ('rule', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.rule')),
+                ('reaction_evidence', models.ManyToManyField(to='epdb.reaction')),
+            ],
+            options={
+                'abstract': False,
+            },
+        ),
         migrations.AddField(
             model_name='edge',
             name='edge_label',
@@ -382,12 +444,11 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('scenario_date', models.CharField(default='No date', max_length=256)),
                 ('scenario_type', models.CharField(default='Not specified', max_length=256)),
-                ('additional_information', models.JSONField(verbose_name='Additional Information')),
-                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Package')),
-                ('parent', models.ForeignKey(default=None, null=True, on_delete=django.db.models.deletion.CASCADE, to='epdb.scenario')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Package')),
             ],
             options={
                 'abstract': False,
@@ -437,14 +498,17 @@ class Migration(migrations.Migration):
                 ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
                 ('name', models.TextField(default='no name', verbose_name='Name')),
                 ('description', models.TextField(default='no description', verbose_name='Descriptions')),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
                 ('kv', models.JSONField(blank=True, default=dict, null=True)),
                 ('public', models.BooleanField(default=False)),
                 ('global_default', models.BooleanField(default=False)),
                 ('max_depth', models.IntegerField(default=5, verbose_name='Setting Max Depth')),
                 ('max_nodes', models.IntegerField(default=30, verbose_name='Setting Max Number of Nodes')),
                 ('model_threshold', models.FloatField(blank=True, default=0.25, null=True, verbose_name='Setting Model Threshold')),
+                ('expansion_scheme', models.CharField(choices=[('BFS', 'Breadth First Search'), ('DFS', 'Depth First Search'), ('GREEDY', 'Greedy')], default='BFS', max_length=20)),
                 ('model', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.epmodel', verbose_name='Setting EPModel')),
-                ('rule_packages', models.ManyToManyField(blank=True, related_name='setting_rule_packages', to='epdb.package', verbose_name='Setting Rule Packages')),
+                ('rule_packages', models.ManyToManyField(blank=True, related_name='setting_rule_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Setting Rule Packages')),
+                ('property_models', models.ManyToManyField(blank=True, related_name='settings', to='epdb.propertypluginmodel', verbose_name='Setting Property Models')),
             ],
             options={
                 'abstract': False,
@@ -461,39 +525,103 @@ class Migration(migrations.Migration):
             field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.setting', verbose_name='The users default settings'),
         ),
         migrations.CreateModel(
-            name='MLRelativeReasoning',
+            name='EnviFormer',
             fields=[
                 ('epmodel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='epdb.epmodel')),
                 ('threshold', models.FloatField(default=0.5)),
-                ('model_status', models.CharField(choices=[('INITIAL', 'Initial'), ('INITIALIZING', 'Model is initializing.'), ('BUILDING', 'Model is building.'), ('BUILT_NOT_EVALUATED', 'Model is built and can be used for predictions, Model is not evaluated yet.'), ('EVALUATING', 'Model is evaluating'), ('FINISHED', 'Model has finished building and evaluation.'), ('ERROR', 'Model has failed.')], default='INITIAL')),
                 ('eval_results', models.JSONField(blank=True, default=dict, null=True)),
-                ('data_packages', models.ManyToManyField(related_name='data_packages', to='epdb.package', verbose_name='Data Packages')),
-                ('eval_packages', models.ManyToManyField(related_name='eval_packages', to='epdb.package', verbose_name='Evaluation Packages')),
-                ('rule_packages', models.ManyToManyField(related_name='rule_packages', to='epdb.package', verbose_name='Rule Packages')),
+                ('multigen_eval', models.BooleanField(default=False)),
+                ('app_domain', models.ForeignKey(blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.applicabilitydomain')),
+                ('data_packages', models.ManyToManyField(related_name='%(app_label)s_%(class)s_data_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Data Packages')),
+                ('eval_packages', models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_eval_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Evaluation Packages')),
+                ('rule_packages', models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_rule_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Rule Packages')),
             ],
             options={
                 'abstract': False,
-                'base_manager_name': 'objects',
             },
             bases=('epdb.epmodel',),
         ),
         migrations.CreateModel(
-            name='ApplicabilityDomain',
+            name='MLRelativeReasoning',
+            fields=[
+                ('epmodel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='epdb.epmodel')),
+                ('threshold', models.FloatField(default=0.5)),
+                ('eval_results', models.JSONField(blank=True, default=dict, null=True)),
+                ('multigen_eval', models.BooleanField(default=False)),
+                ('app_domain', models.ForeignKey(blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.applicabilitydomain')),
+                ('data_packages', models.ManyToManyField(related_name='%(app_label)s_%(class)s_data_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Data Packages')),
+                ('eval_packages', models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_eval_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Evaluation Packages')),
+                ('rule_packages', models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_rule_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Rule Packages')),
+            ],
+            options={
+                'abstract': False,
+            },
+            bases=('epdb.epmodel',),
+        ),
+        migrations.AddField(
+            model_name='applicabilitydomain',
+            name='model',
+            field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.mlrelativereasoning'),
+        ),
+        migrations.AddField(
+            model_name='propertypluginmodel',
+            name='app_domain',
+            field=models.ForeignKey(blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.applicabilitydomain'),
+        ),
+        migrations.AddField(
+            model_name='propertypluginmodel',
+            name='data_packages',
+            field=models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_data_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Data Packages'),
+        ),
+        migrations.AddField(
+            model_name='propertypluginmodel',
+            name='eval_packages',
+            field=models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_eval_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Evaluation Packages'),
+        ),
+        migrations.AddField(
+            model_name='propertypluginmodel',
+            name='rule_packages',
+            field=models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_rule_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Rule Packages'),
+        ),
+        migrations.CreateModel(
+            name='RuleBasedRelativeReasoning',
+            fields=[
+                ('epmodel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='epdb.epmodel')),
+                ('threshold', models.FloatField(default=0.5)),
+                ('eval_results', models.JSONField(blank=True, default=dict, null=True)),
+                ('multigen_eval', models.BooleanField(default=False)),
+                ('min_count', models.IntegerField(default=10)),
+                ('max_count', models.IntegerField(default=0)),
+                ('app_domain', models.ForeignKey(blank=True, default=None, null=True, on_delete=django.db.models.deletion.SET_NULL, to='epdb.applicabilitydomain')),
+                ('data_packages', models.ManyToManyField(related_name='%(app_label)s_%(class)s_data_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Data Packages')),
+                ('eval_packages', models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_eval_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Evaluation Packages')),
+                ('rule_packages', models.ManyToManyField(blank=True, related_name='%(app_label)s_%(class)s_rule_packages', to=settings.EPDB_PACKAGE_MODEL, verbose_name='Rule Packages')),
+            ],
+            options={
+                'abstract': False,
+            },
+            bases=('epdb.epmodel',),
+        ),
+        migrations.CreateModel(
+            name='ExternalIdentifier',
             fields=[
                 ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                 ('created', model_utils.fields.AutoCreatedField(default=django.utils.timezone.now, editable=False, verbose_name='created')),
                 ('modified', model_utils.fields.AutoLastModifiedField(default=django.utils.timezone.now, editable=False, verbose_name='modified')),
-                ('uuid', models.UUIDField(default=uuid.uuid4, unique=True, verbose_name='UUID of this object')),
-                ('name', models.TextField(default='no name', verbose_name='Name')),
-                ('description', models.TextField(default='no description', verbose_name='Descriptions')),
-                ('kv', models.JSONField(blank=True, default=dict, null=True)),
-                ('num_neighbours', models.FloatField(default=5)),
-                ('reliability_threshold', models.FloatField(default=0.5)),
-                ('local_compatibilty_threshold', models.FloatField(default=0.5)),
-                ('model', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.mlrelativereasoning')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, editable=False, unique=True)),
+                ('object_id', models.IntegerField()),
+                ('identifier_value', models.CharField(max_length=255, verbose_name='Identifier Value')),
+                ('url', models.URLField(blank=True, null=True, verbose_name='Direct URL')),
+                ('is_primary', models.BooleanField(default=False, help_text='Mark this as the primary identifier for this database', verbose_name='Is Primary')),
+                ('content_type', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='contenttypes.contenttype')),
+                ('database', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.externaldatabase', verbose_name='External Database')),
             ],
             options={
-                'abstract': False,
+                'verbose_name': 'External Identifier',
+                'verbose_name_plural': 'External Identifiers',
+                'db_table': 'epdb_external_identifier',
+                'indexes': [models.Index(fields=['content_type', 'object_id'], name='epdb_extern_content_b76813_idx'), models.Index(fields=['database', 'identifier_value'], name='epdb_extern_databas_486422_idx')],
+                'unique_together': {('content_type', 'object_id', 'database', 'identifier_value')},
             },
         ),
         migrations.CreateModel(
@@ -552,13 +680,32 @@ class Migration(migrations.Migration):
             name='compound',
             unique_together={('uuid', 'package')},
         ),
+        migrations.CreateModel(
+            name='AdditionalInformation',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('uuid', models.UUIDField(default=uuid.uuid4, editable=False, unique=True)),
+                ('url', models.TextField(null=True, unique=True, verbose_name='URL')),
+                ('kv', models.JSONField(blank=True, default=dict, null=True)),
+                ('type', models.TextField(verbose_name='Additional Information Type')),
+                ('data', models.JSONField(blank=True, default=dict, null=True)),
+                ('object_id', models.PositiveBigIntegerField(blank=True, null=True)),
+                ('content_type', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to='contenttypes.contenttype')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Package')),
+                ('scenario', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='scenario_additional_information', to='epdb.scenario')),
+            ],
+            options={
+                'indexes': [models.Index(fields=['type'], name='epdb_additi_type_394349_idx'), models.Index(fields=['scenario', 'type'], name='epdb_additi_scenari_a59edf_idx'), models.Index(fields=['content_type', 'object_id'], name='epdb_additi_content_44d4b4_idx'), models.Index(fields=['scenario', 'content_type', 'object_id'], name='epdb_additi_scenari_ef2bf5_idx')],
+                'constraints': [models.CheckConstraint(condition=models.Q(models.Q(('content_type__isnull', True), ('object_id__isnull', True)), models.Q(('content_type__isnull', False), ('object_id__isnull', False)), _connector='OR'), name='ck_addinfo_gfk_pair'), models.CheckConstraint(condition=models.Q(('scenario__isnull', False), ('content_type__isnull', False), _connector='OR'), name='ck_addinfo_not_both_null')],
+            },
+        ),
         migrations.CreateModel(
             name='GroupPackagePermission',
             fields=[
                 ('permission_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, to='epdb.permission')),
                 ('uuid', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False, verbose_name='UUID of this object')),
                 ('group', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.group', verbose_name='Permission to')),
-                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Permission on')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Permission on')),
             ],
             options={
                 'unique_together': {('package', 'group')},
@@ -570,7 +717,7 @@ class Migration(migrations.Migration):
             fields=[
                 ('permission_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, to='epdb.permission')),
                 ('uuid', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False, verbose_name='UUID of this object')),
-                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='epdb.package', verbose_name='Permission on')),
+                ('package', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.EPDB_PACKAGE_MODEL, verbose_name='Permission on')),
                 ('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL, verbose_name='Permission to')),
             ],
             options={

epdb/migrations/0020_alter_compoundstructure_options_and_more.py

@@ -0,0 +1,65 @@
+# Generated by Django 5.2.7 on 2026-03-09 10:41
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+def populate_polymorphic_ctype(apps, schema_editor):
+    ContentType = apps.get_model("contenttypes", "ContentType")
+    Compound = apps.get_model("epdb", "Compound")
+    CompoundStructure = apps.get_model("epdb", "CompoundStructure")
+
+    # Update Compound records
+    compound_ct = ContentType.objects.get_for_model(Compound)
+    Compound.objects.filter(polymorphic_ctype__isnull=True).update(polymorphic_ctype=compound_ct)
+
+    # Update CompoundStructure records
+    compound_structure_ct = ContentType.objects.get_for_model(CompoundStructure)
+    CompoundStructure.objects.filter(polymorphic_ctype__isnull=True).update(
+        polymorphic_ctype=compound_structure_ct
+    )
+
+
+def reverse_populate_polymorphic_ctype(apps, schema_editor):
+    Compound = apps.get_model("epdb", "Compound")
+    CompoundStructure = apps.get_model("epdb", "CompoundStructure")
+
+    Compound.objects.all().update(polymorphic_ctype=None)
+    CompoundStructure.objects.all().update(polymorphic_ctype=None)
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("contenttypes", "0002_remove_content_type_name"),
+        ("epdb", "0019_remove_scenario_additional_information_and_more"),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name="compoundstructure",
+            options={"base_manager_name": "objects"},
+        ),
+        migrations.AddField(
+            model_name="compound",
+            name="polymorphic_ctype",
+            field=models.ForeignKey(
+                editable=False,
+                null=True,
+                on_delete=django.db.models.deletion.CASCADE,
+                related_name="polymorphic_%(app_label)s.%(class)s_set+",
+                to="contenttypes.contenttype",
+            ),
+        ),
+        migrations.AddField(
+            model_name="compoundstructure",
+            name="polymorphic_ctype",
+            field=models.ForeignKey(
+                editable=False,
+                null=True,
+                on_delete=django.db.models.deletion.CASCADE,
+                related_name="polymorphic_%(app_label)s.%(class)s_set+",
+                to="contenttypes.contenttype",
+            ),
+        ),
+        migrations.RunPython(populate_polymorphic_ctype, reverse_populate_polymorphic_ctype),
+    ]

epdb/tasks.py

@@ -7,10 +7,21 @@ from uuid import uuid4
 from celery import shared_task
 from celery.utils.functional import LRUCache
 from django.conf import settings as s
+from django.core.mail import EmailMultiAlternatives
 from django.utils import timezone
 
 from epdb.logic import SPathway
-from epdb.models import Edge, EPModel, JobLog, Node, Pathway, Rule, Setting, User
+from epdb.models import (
+    AdditionalInformation,
+    Edge,
+    EPModel,
+    JobLog,
+    Node,
+    Pathway,
+    Rule,
+    Setting,
+    User,
+)
 from utilities.chem import FormatConverter
 
 logger = logging.getLogger(__name__)
@@ -65,15 +76,39 @@ def mul(a, b):
 
 
 @shared_task(queue="predict")
-def predict_simple(model_pk: int, smiles: str):
+def predict_simple(model_pk: int, smiles: str, *args, **kwargs):
     mod = get_ml_model(model_pk)
-    res = mod.predict(smiles)
+    res = mod.predict(smiles, *args, **kwargs)
     return res
 
 
 @shared_task(queue="background")
 def send_registration_mail(user_pk: int):
-    pass
+    u = User.objects.get(id=user_pk)
+
+    tpl = """Welcome {username}!,
+
+Thank you for your interest in enviPath.
+
+The public system is intended for non-commercial use only.
+We will review your account details and usually activate your account within 24 hours.
+Once activated, you will be notified by email.
+
+If we have any questions, we will contact you at this email address.
+
+Best regards,
+
+enviPath team"""
+
+    msg = EmailMultiAlternatives(
+        "Your enviPath account",
+        tpl.format(username=u.username),
+        "admin@envipath.org",
+        [u.email],
+        bcc=["admin@envipath.org"],
+    )
+
+    msg.send(fail_silently=False)
 
 
 @shared_task(bind=True, queue="model")
@@ -204,9 +239,28 @@ def predict(
     if JobLog.objects.filter(task_id=self.request.id).exists():
         JobLog.objects.filter(task_id=self.request.id).update(status="SUCCESS", task_result=pw.url)
 
+    # dispatch property job
+    compute_properties.delay(pw_pk, pred_setting_pk)
+
     return pw.url
 
 
+@shared_task(bind=True, queue="background")
+def compute_properties(self, pathway_pk: int, setting_pk: int):
+    pw = Pathway.objects.get(id=pathway_pk)
+    setting = Setting.objects.get(id=setting_pk)
+
+    nodes = [n for n in pw.nodes]
+    smiles = [n.default_node_label.smiles for n in nodes]
+
+    for prop_mod in setting.property_models.all():
+        if prop_mod.instance().is_heavy():
+            rr = prop_mod.predict_batch(smiles)
+            for idx, pred in enumerate(rr.result):
+                n = nodes[idx]
+                _ = AdditionalInformation.create(pw.package, ai=pred, content_object=n)
+
+
 @shared_task(bind=True, queue="background")
 def identify_missing_rules(
     self,
@@ -395,7 +449,7 @@ def batch_predict(
     standardized_substrates_and_smiles = []
     for substrate in substrate_and_names:
         try:
-            stand_smiles = FormatConverter.standardize(substrate[0])
+            stand_smiles = FormatConverter.standardize(substrate[0], remove_stereo=True)
             standardized_substrates_and_smiles.append([stand_smiles, substrate[1]])
         except ValueError:
             raise ValueError(

epdb/urls.py

@@ -209,5 +209,4 @@ urlpatterns = [
     re_path(r"^contact$", v.static_contact_support, name="contact_support"),
     re_path(r"^careers$", v.static_careers, name="careers"),
     re_path(r"^cite$", v.static_cite, name="cite"),
-    re_path(r"^legal$", v.static_legal, name="legal"),
 ]

migration/views.py

@@ -76,9 +76,7 @@ def migration(request):
                 open(s.BASE_DIR / "fixtures" / "migration_status_per_rule.json")
             )
         else:
-            BBD = Package.objects.get(
-                url="http://localhost:8000/package/32de3cf4-e3e6-4168-956e-32fa5ddb0ce1"
-            )
+            BBD = Package.objects.get(uuid="32de3cf4-e3e6-4168-956e-32fa5ddb0ce1")
             ALL_SMILES = [
                 cs.smiles for cs in CompoundStructure.objects.filter(compound__package=BBD)
             ]
@@ -147,7 +145,7 @@ def migration_detail(request, package_uuid, rule_uuid):
     if request.method == "GET":
         context = get_base_context(request)
 
-        BBD = Package.objects.get(name="EAWAG-BBD")
+        BBD = Package.objects.get(uuid="32de3cf4-e3e6-4168-956e-32fa5ddb0ce1")
         STRUCTURES = CompoundStructure.objects.filter(compound__package=BBD)
         rule = Rule.objects.get(package=BBD, uuid=rule_uuid)
 

package.json

@@ -8,14 +8,14 @@
     "build": "tailwindcss -i static/css/input.css -o static/css/output.css --minify"
   },
   "devDependencies": {
-    "@tailwindcss/cli": "^4.1.16",
-    "@tailwindcss/postcss": "^4.1.16",
-    "daisyui": "^5.4.3",
+    "@tailwindcss/cli": "^4.1.18",
+    "@tailwindcss/postcss": "^4.1.18",
+    "daisyui": "^5.5.14",
     "postcss": "^8.5.6",
-    "prettier": "^3.6.2",
+    "prettier": "^3.7.4",
     "prettier-plugin-jinja-template": "^2.1.0",
-    "prettier-plugin-tailwindcss": "^0.7.1",
-    "tailwindcss": "^4.1.16"
+    "prettier-plugin-tailwindcss": "^0.7.2",
+    "tailwindcss": "^4.1.18"
   },
   "keywords": [
     "django",

pepper/__init__.py

@@ -0,0 +1,361 @@
+import logging
+import math
+import os
+import pickle
+from datetime import datetime
+from typing import Any, List, Optional
+
+import polars as pl
+
+from pydantic import computed_field
+from sklearn.metrics import (
+    mean_absolute_error,
+    mean_squared_error,
+    r2_score,
+    root_mean_squared_error,
+)
+from sklearn.model_selection import ShuffleSplit
+
+# Once stable these will be exposed by enviPy-plugins lib
+from envipy_additional_information import register  # noqa: I001
+from bridge.contracts import Property, PropertyType  # noqa: I001
+from bridge.dto import (
+    BuildResult,
+    EnviPyDTO,
+    EvaluationResult,
+    PredictedProperty,
+    RunResult,
+)  # noqa: I001
+
+from .impl.pepper import Pepper  # noqa: I001
+
+logger = logging.getLogger(__name__)
+
+
+@register("pepperprediction")
+class PepperPrediction(PredictedProperty):
+    mean: float | None
+    std: float | None
+    log_mean: float | None
+    log_std: float | None
+
+    @computed_field
+    @property
+    def svg(self, xscale="linear", quantiles=(0.01, 0.99), n_points=2000) -> Optional[str]:
+        import io
+
+        import matplotlib.patches as mpatches
+        import numpy as np
+        from matplotlib import pyplot as plt
+        from scipy import stats
+
+        """
+        Plot the lognormal distribution of chemical half-lives where parameters are
+        given on a base-10 log scale: log10(half-life) ~ Normal(mu_log10, sigma_log10^2).
+
+        Shades:
+          - x < a in green (Non-persistent)
+          - a <= x <= b in yellow (Persistent)
+          - x > b in red (Very persistent)
+
+        Legend shows the shaded color and the probability mass in each region.
+        """
+
+        sigma_log10 = self.log_std
+        mu_log10 = self.log_mean
+
+        if sigma_log10 <= 0:
+            raise ValueError("sigma_log10 must be > 0")
+        # Persistent and Very Persistent thresholds in days from REACH (https://doi.org/10.26434/chemrxiv-2025-xmslf)
+        p = 120
+        vp = 180
+
+        # Convert base-10 log parameters to natural-log parameters for SciPy's lognorm
+        ln10 = np.log(10.0)
+        mu_ln = mu_log10 * ln10
+        sigma_ln = sigma_log10 * ln10
+
+        # SciPy parameterization: lognorm(s=sigma_ln, scale=exp(mu_ln))
+        dist = stats.lognorm(s=sigma_ln, scale=np.exp(mu_ln))
+
+        # Exact probabilities
+        p_green = dist.cdf(p)  # P(X < p) prob not persistent
+        p_yellow = 1.0 - dist.cdf(p)  # P (X > p) prob persistent
+        p_red = 1.0 - dist.cdf(vp)  # P(X > vp) prob very persistent
+
+        # Plotting range
+        q_low, q_high = dist.ppf(quantiles)
+        x_min = max(1e-12, min(q_low, p) * 0.9)
+        x_max = max(q_high, vp) * 1.1
+
+        # Build x-grid (linear days axis)
+        if xscale == "log":
+            x = np.logspace(np.log10(x_min), np.log10(x_max), n_points)
+        else:
+            x = np.linspace(x_min, x_max, n_points)
+        y = dist.pdf(x)
+
+        # Masks for shading
+        mask_green = x < p
+        mask_yellow = (x >= p) & (x <= vp)
+        mask_red = x > vp
+
+        # Plot
+        fig, ax = plt.subplots(figsize=(9, 5.5))
+        ax.plot(x, y, color="#1f4e79", lw=2, label="Lognormal PDF")
+
+        if np.any(mask_green):
+            ax.fill_between(x[mask_green], y[mask_green], 0, color="tab:green", alpha=0.3)
+        if np.any(mask_yellow):
+            ax.fill_between(x[mask_yellow], y[mask_yellow], 0, color="gold", alpha=0.35)
+        if np.any(mask_red):
+            ax.fill_between(x[mask_red], y[mask_red], 0, color="tab:red", alpha=0.3)
+
+        # Threshold lines
+        ax.axvline(p, color="gray", ls="--", lw=1)
+        ax.axvline(vp, color="gray", ls="--", lw=1)
+
+        # Labels & title
+        ax.set_title(
+            f"Half-life Distribution (Lognormal)\nlog10 parameters: μ={mu_log10:g}, σ={sigma_log10:g}"
+        )
+        ax.set_xlabel("Half-life (days)")
+        ax.set_ylabel("Probability density")
+        ax.grid(True, alpha=0.25)
+
+        if xscale == "log":
+            ax.set_xscale("log")  # not used in this example, but supported
+
+        # Legend with probabilities
+        patches = [
+            mpatches.Patch(
+                color="tab:green",
+                alpha=0.3,
+                label=f"Non-persistent (<{p:g} d): {p_green:.2%}",
+            ),
+            mpatches.Patch(
+                color="gold",
+                alpha=0.35,
+                label=f"Persistent ({p:g}–{vp:g} d): {p_yellow:.2%}",
+            ),
+            mpatches.Patch(
+                color="tab:red",
+                alpha=0.3,
+                label=f"Very persistent (>{vp:g} d): {p_red:.2%}",
+            ),
+        ]
+        ax.legend(handles=patches, frameon=True)
+
+        plt.tight_layout()
+
+        # --- Export to SVG string ---
+        buf = io.StringIO()
+        fig.savefig(buf, format="svg", bbox_inches="tight")
+        svg = buf.getvalue()
+        plt.close(fig)
+        buf.close()
+
+        return svg
+
+
+class PEPPER(Property):
+    def identifier(self) -> str:
+        return "pepper"
+
+    def display(self) -> str:
+        return "PEPPER"
+
+    def name(self) -> str:
+        return "Predict Environmental Pollutant PERsistence"
+
+    def requires_rule_packages(self) -> bool:
+        return False
+
+    def requires_data_packages(self) -> bool:
+        return True
+
+    def get_type(self) -> PropertyType:
+        return PropertyType.HEAVY
+
+    def generate_dataset(self, eP: EnviPyDTO) -> pl.DataFrame:
+        """
+        Generates a dataset in the form of a Polars DataFrame containing compound information, including
+        SMILES strings and logarithmic values of degradation half-lives (dt50).
+
+        The dataset is built by iterating over a list of compounds, standardizing SMILES strings, and
+        calculating the logarithmic mean of the half-life intervals for different environmental scenarios
+        associated with each compound.
+
+        The resulting DataFrame will only include unique rows based on SMILES and logarithmic half-life
+        values.
+
+        Parameters:
+            eP (EnviPyDTO): An object that provides access to compound data and utility functions for
+            standardization and retrieval of half-life information.
+
+        Returns:
+            pl.DataFrame: The resulting dataset with unique rows containing compound structure identifiers,
+            standardized SMILES strings, and logarithmic half-life values.
+
+        Raises:
+            Exception: Exceptions are caught and logged during data processing, specifically when retrieving
+            half-life information.
+
+        Note:
+            - The logarithmic mean is calculated from the start and end intervals of the dt50 (half-life).
+            - Compounds not associated with any half-life data are skipped, and errors encountered during processing
+              are logged without halting the execution.
+        """
+        columns = ["structure_id", "smiles", "dt50_log"]
+        rows = []
+
+        for c in eP.get_compounds():
+            hls = c.half_lifes()
+
+            if len(hls):
+                stand_smiles = eP.standardize(c.smiles, remove_stereo=True)
+                for scenario, half_lives in hls.items():
+                    for h in half_lives:
+                        # In the original Pepper code they take the mean of the start and end interval.
+                        half_mean = (h.dt50.start + h.dt50.end) / 2
+                        rows.append([str(c.url), stand_smiles, math.log10(half_mean)])
+
+        df = pl.DataFrame(data=rows, schema=columns, orient="row", infer_schema_length=None)
+
+        df = df.unique(subset=["smiles", "dt50_log"], keep="any", maintain_order=False)
+
+        return df
+
+    def save_dataset(self, df: pl.DataFrame, path: str):
+        with open(path, "wb") as fh:
+            pickle.dump(df, fh)
+
+    def load_dataset(self, path: str) -> pl.DataFrame:
+        with open(path, "rb") as fh:
+            return pickle.load(fh)
+
+    def build(self, eP: EnviPyDTO, *args, **kwargs) -> BuildResult | None:
+        logger.info(f"Start building PEPPER {eP.get_context().uuid}")
+        df = self.generate_dataset(eP)
+
+        if df.shape[0] == 0:
+            raise ValueError("No data found for building model")
+
+        p = Pepper()
+
+        p, train_ds = p.train_model(df)
+
+        ds_store_path = os.path.join(
+            eP.get_context().work_dir, f"pepper_ds_{eP.get_context().uuid}.pkl"
+        )
+        self.save_dataset(train_ds, ds_store_path)
+
+        model_store_path = os.path.join(
+            eP.get_context().work_dir, f"pepper_{eP.get_context().uuid}.pkl"
+        )
+        p.save_model(model_store_path)
+        logger.info(f"Finished building PEPPER {eP.get_context().uuid}")
+
+    def run(self, eP: EnviPyDTO, *args, **kwargs) -> RunResult:
+        load_path = os.path.join(eP.get_context().work_dir, f"pepper_{eP.get_context().uuid}.pkl")
+
+        p = Pepper.load_model(load_path)
+
+        X_new = [c.smiles for c in eP.get_compounds()]
+
+        predictions = p.predict_batch(X_new)
+
+        results = []
+
+        for p in zip(*predictions):
+            if p[0] is None or p[1] is None:
+                result = {"log_mean": None, "mean": None, "log_std": None, "std": None, "svg": None}
+            else:
+                result = {
+                    "log_mean": p[0],
+                    "mean": 10 ** p[0],
+                    "log_std": p[1],
+                    "std": 10 ** p[1],
+                }
+
+            results.append(PepperPrediction(**result))
+
+        rr = RunResult(
+            producer=eP.get_context().url,
+            description=f"Generated at {datetime.now()}",
+            result=results,
+        )
+
+        return rr
+
+    def evaluate(self, eP: EnviPyDTO, *args, **kwargs) -> EvaluationResult | None:
+        logger.info(f"Start evaluating PEPPER {eP.get_context().uuid}")
+        load_path = os.path.join(eP.get_context().work_dir, f"pepper_{eP.get_context().uuid}.pkl")
+
+        p = Pepper.load_model(load_path)
+
+        df = self.generate_dataset(eP)
+        ds = p.preprocess_data(df)
+
+        y_pred = p.predict_batch(ds["smiles"])
+
+        # We only need the mean
+        if isinstance(y_pred, tuple):
+            y_pred = y_pred[0]
+
+        res = self.eval_stats(ds["dt50_bayesian_mean"], y_pred)
+
+        logger.info(f"Finished evaluating PEPPER {eP.get_context().uuid}")
+        return EvaluationResult(data=res)
+
+    def build_and_evaluate(self, eP: EnviPyDTO, *args, **kwargs) -> EvaluationResult | None:
+        logger.info(f"Start evaluating PEPPER {eP.get_context().uuid}")
+        ds_load_path = os.path.join(
+            eP.get_context().work_dir, f"pepper_ds_{eP.get_context().uuid}.pkl"
+        )
+        ds = self.load_dataset(ds_load_path)
+
+        n_splits = kwargs.get("n_splits", 20)
+        shuff = ShuffleSplit(n_splits=n_splits, test_size=0.1, random_state=42)
+
+        fold_metrics: List[dict[str, Any]] = []
+        for split_id, (train_index, test_index) in enumerate(shuff.split(ds)):
+            logger.info(f"Evaluation fold {split_id}/{n_splits} PEPPER {eP.get_context().uuid}")
+            train = ds[train_index]
+            test = ds[test_index]
+            model = Pepper()
+            model.train_model(train, preprocess=False)
+
+            features = test[model.descriptors.get_descriptor_names()].rows()
+            y_pred = model.predict_batch(features, is_smiles=False)
+
+            # We only need the mean for eval statistics but mean, std can be returned
+            if isinstance(y_pred, tuple) or isinstance(y_pred, list):
+                y_pred = y_pred[0]
+
+            # Remove None if they occur
+            y_true_filtered, y_pred_filtered = [], []
+            for t, p in zip(test["dt50_bayesian_mean"], y_pred):
+                if p is None:
+                    continue
+                y_true_filtered.append(t)
+                y_pred_filtered.append(p)
+
+            if len(y_true_filtered) == 0:
+                print("Skipping empty fold")
+                continue
+
+            fold_metrics.append(self.eval_stats(y_true_filtered, y_pred_filtered))
+
+        logger.info(f"Finished evaluating PEPPER {eP.get_context().uuid}")
+        return EvaluationResult(data=fold_metrics)
+
+    @staticmethod
+    def eval_stats(y_true, y_pred) -> dict[str, float]:
+        scores_dic = {
+            "r2": r2_score(y_true, y_pred),
+            "mse": mean_squared_error(y_true, y_pred),
+            "rmse": root_mean_squared_error(y_true, y_pred),
+            "mae": mean_absolute_error(y_true, y_pred),
+        }
+        return scores_dic

pepper/impl/bayesian.py

@@ -0,0 +1,196 @@
+import emcee
+import numpy as np
+from scipy.stats import lognorm, norm
+
+
+class Bayesian:
+    def __init__(self, y, comment_list=None):
+        if comment_list is None:
+            comment_list = []
+        self.y = y
+        self.comment_list = comment_list
+        # LOQ default settings
+        self.LOQ_lower = -1  # (2.4 hours)
+        self.LOQ_upper = 3  # 1000 days
+        # prior default settings
+        self.prior_mu_mean = 1.5
+        self.prior_mu_std = 2
+        self.prior_sigma_mean = 0.4
+        self.prior_sigma_std = 0.4
+        self.lower_limit_sigma = 0.2
+        # EMCEE defaults
+        self.nwalkers = 10
+        self.iterations = 2000
+        self.burn_in = 100
+        ndim = 2  # number of dimensions (mean, std)
+        # backend = emcee.backends.HDFBackend("backend.h5")
+        # backend.reset(self.nwalkers, ndim)
+        self.sampler = emcee.EnsembleSampler(self.nwalkers, ndim, self.logPosterior)
+        self.posterior_mu = None
+        self.posterior_sigma = None
+
+    def get_censored_values_only(self):
+        censored_values = []
+        for i, comment in enumerate(self.comment_list):
+            if comment in ["<", ">"]:
+                censored_values.append(self.y[i])
+            elif self.y[i] > self.LOQ_upper or self.y[i] < self.LOQ_lower:
+                censored_values.append(self.y[i])
+        return censored_values
+
+    # Class functions
+    def determine_LOQ(self):
+        """
+        Determines if the LOQ is upper or lower, and the value (if not default)
+        :return: upper_LOQ , lower_LOQ
+        """
+
+        censored_values = self.get_censored_values_only()
+
+        # Find upper LOQ
+        upper_LOQ = np.nan
+        # bigger than global LOQ
+        if max(self.y) >= self.LOQ_upper:
+            upper_LOQ = self.LOQ_upper
+        # case if exactly 365 days
+        elif max(self.y) == 2.562:  # 365 days
+            upper_LOQ = 2.562
+            self.LOQ_upper = upper_LOQ
+        # case if "bigger than" indication in comments
+        elif ">" in self.comment_list:
+            i = 0
+            while i < len(self.y):
+                if self.y[i] == min(censored_values) and self.comment_list[i] == ">":
+                    self.LOQ_upper = self.y[i]
+                    break
+                i += 1
+
+        # Find lower LOQ
+        lower_LOQ = np.nan
+        # smaller than global LOQ
+        if min(self.y) <= self.LOQ_lower:
+            lower_LOQ = self.LOQ_lower
+        # case if exactly 1 day
+        elif min(self.y) == 0:  # 1 day
+            lower_LOQ = 0
+            self.LOQ_lower = 0
+        # case if "smaller than" indication in comments
+        elif "<" in self.comment_list:
+            i = 0
+            while i < len(self.y):
+                if self.y[i] == max(censored_values) and self.comment_list[i] == "<":
+                    self.LOQ_lower = self.y[i]
+                    break
+                i += 1
+        return upper_LOQ, lower_LOQ
+
+    def logLikelihood(self, theta, sigma):
+        """
+        Likelihood function (the probability of a dataset (mean, std) given the model parameters)
+        Convert not censored observations into type ’numeric’
+        :param theta: mean half-life value to be evaluated
+        :param sigma: std half-life value to be evaluated
+        :return: log_likelihood
+        """
+        upper_LOQ, lower_LOQ = self.determine_LOQ()
+
+        n_censored_upper = 0
+        n_censored_lower = 0
+        y_not_cen = []
+
+        if np.isnan(upper_LOQ) and np.isnan(lower_LOQ):
+            y_not_cen = self.y
+        else:
+            for i in self.y:
+                if np.isnan(upper_LOQ) and i >= upper_LOQ:  # censor above threshold
+                    n_censored_upper += 1
+                if np.isnan(lower_LOQ) and i <= lower_LOQ:  # censor below threshold
+                    n_censored_lower += 1
+                else:  # do not censor
+                    y_not_cen.append(i)
+
+        LL_left_cen = 0
+        LL_right_cen = 0
+        LL_not_cen = 0
+
+        # likelihood for not censored observations
+        if n_censored_lower > 0:  # loglikelihood for left censored observations
+            LL_left_cen = n_censored_lower * norm.logcdf(
+                lower_LOQ, loc=theta, scale=sigma
+            )  # cumulative distribution function CDF
+
+        if n_censored_upper > 0:  # loglikelihood for right censored observations
+            LL_right_cen = n_censored_upper * norm.logsf(
+                upper_LOQ, loc=theta, scale=sigma
+            )  # survival function (1-CDF)
+
+        if len(y_not_cen) > 0:  # loglikelihood for uncensored values
+            LL_not_cen = sum(
+                norm.logpdf(y_not_cen, loc=theta, scale=sigma)
+            )  # probability density function PDF
+
+        return LL_left_cen + LL_not_cen + LL_right_cen
+
+    def get_prior_probability_sigma(self, sigma):
+        # convert mean and sd to logspace parameters, to see this formula check
+        # https://en.wikipedia.org/wiki/Log-normal_distribution under Method of moments section
+        temp = 1 + (self.prior_sigma_std / self.prior_sigma_mean) ** 2
+        meanlog = self.prior_sigma_mean / np.sqrt(temp)
+        sdlog = np.sqrt(np.log(temp))
+        # calculate of logpdf of sigma
+        norm_pdf_sigma = lognorm.logpdf(sigma, s=sdlog, loc=self.lower_limit_sigma, scale=meanlog)
+        return norm_pdf_sigma
+
+    def get_prior_probability_theta(self, theta):
+        norm_pdf_theta = norm.logpdf(theta, loc=self.prior_mu_mean, scale=self.prior_mu_std)
+        return norm_pdf_theta
+
+    def logPrior(self, par):
+        """
+        Obtain prior loglikelihood of [theta, sigma]
+        :param par: par = [theta,sigma]
+        :return: loglikelihood
+        """
+        # calculate the mean and standard deviation in the log-space
+        norm_pdf_mean = self.get_prior_probability_theta(par[0])
+        norm_pdf_std = self.get_prior_probability_sigma(par[1])
+        log_norm_pdf = [norm_pdf_mean, norm_pdf_std]
+        return sum(log_norm_pdf)
+
+    def logPosterior(self, par):
+        """
+        Obtain posterior loglikelihood
+        :param par: [theta, sigma]
+        :return: posterior loglikelihood
+        """
+        logpri = self.logPrior(par)
+        if not np.isfinite(logpri):
+            return -np.inf
+        loglikelihood = self.logLikelihood(par[0], par[1])
+        return logpri + loglikelihood
+
+    def get_posterior_distribution(self):
+        """
+        Sample posterior distribution and get median of mean and std samples
+        :return: posterior half-life mean and std
+        """
+        if self.posterior_mu:
+            return self.posterior_mu, self.posterior_sigma
+
+        # Sampler parameters
+        ndim = 2  # number of dimensions (mean,std)
+        p0 = abs(np.random.randn(self.nwalkers, ndim))  # only positive starting numbers (for std)
+
+        # Sample distribution
+        self.sampler.run_mcmc(p0, self.iterations)
+        # get chain and log_prob in one-dimensional array (merged chains with burn-in)
+        samples = self.sampler.get_chain(flat=True, discard=100)
+        # get median mean and std
+        self.posterior_mu = np.median(samples[:, 0])
+        self.posterior_sigma = np.median(samples[:, 1])
+        return self.posterior_mu, self.posterior_sigma
+
+
+# Utility functions
+def get_normal_distribution(x, mu, sig):
+    return np.exp(-np.power(x - mu, 2.0) / (2 * np.power(sig, 2.0)))

pepper/impl/config/regressor_settings_singlevalue_soil_paper_GPR_optimized.yml

@@ -0,0 +1,11 @@
+GPR:
+  name: Gaussian Process Regressor
+  regressor: GaussianProcessRegressor
+  regressor_params:
+    normalize_y: True
+    n_restarts_optimizer: 0
+    kernel: "ConstantKernel(1.0, (1e-3, 1e3)) * Matern(length_scale=2.5, length_scale_bounds=(1e-3, 1e3), nu=0.5)"
+  feature_reduction_method: None
+  feature_reduction_parameters:
+    pca:
+      n_components: 34

pepper/impl/descriptors.py

@@ -0,0 +1,60 @@
+from abc import ABC, abstractmethod
+from typing import List
+
+from mordred import Calculator, descriptors
+from padelpy import from_smiles
+from rdkit import Chem
+
+
+class Descriptor(ABC):
+    @abstractmethod
+    def get_molecule_descriptors(self, molecule: str) -> List[float | int] | None:
+        pass
+
+    @abstractmethod
+    def get_descriptor_names(self) -> List[str]:
+        pass
+
+
+class Mordred(Descriptor):
+    calc = Calculator(descriptors, ignore_3D=True)
+
+    def get_molecule_descriptors(self, molecule: str) -> List[float | int] | None:
+        mol = Chem.MolFromSmiles(molecule)
+        res = list(self.calc(mol))
+        return res
+
+    def get_descriptor_names(self) -> List[str]:
+        return [f"Mordred_{i}" for i in range(len(self.calc.descriptors))]
+
+
+class PaDEL(Descriptor):
+    calc = Calculator(descriptors)
+
+    def get_molecule_descriptors(self, molecule: str) -> List[float | int] | None:
+        try:
+            padel_descriptors = from_smiles(molecule, threads=1)
+        except RuntimeError:
+            return []
+
+        formatted = []
+        for k, v in padel_descriptors.items():
+            try:
+                formatted.append(float(v))
+            except ValueError:
+                formatted.append(0.0)
+
+        return formatted
+
+    def get_descriptor_names(self) -> List[str]:
+        return [f"PaDEL_{i}" for i in range(1875)]
+
+
+if __name__ == "__main__":
+    mol = "CC1=CC(O)=CC=C1[N+](=O)[O-]"
+
+    m = Mordred()
+    print(list(m.get_molecule_descriptors(mol)))
+
+    p = PaDEL()
+    print(list(p.get_molecule_descriptors(mol)))

pepper/impl/pepper.py

@@ -0,0 +1,329 @@
+import importlib.resources
+import logging
+import math
+import os
+import pickle
+from collections import defaultdict
+from typing import List
+
+import numpy as np
+import polars as pl
+import yaml
+from joblib import Parallel, delayed
+from scipy.cluster import hierarchy
+from scipy.spatial.distance import squareform
+from scipy.stats import spearmanr
+from sklearn.feature_selection import VarianceThreshold
+from sklearn.gaussian_process import GaussianProcessRegressor
+from sklearn.pipeline import Pipeline
+from sklearn.preprocessing import FunctionTransformer, MinMaxScaler
+
+from .bayesian import Bayesian
+from .descriptors import Mordred
+
+
+class Pepper:
+    def __init__(self, config_path=None, random_state=42):
+        self.random_state = random_state
+        if config_path is None:
+            config_path = importlib.resources.files("pepper.impl.config").joinpath(
+                "regressor_settings_singlevalue_soil_paper_GPR_optimized.yml"
+            )
+        with open(config_path, "r") as file:
+            regressor_settings = yaml.safe_load(file)
+        if len(regressor_settings) > 1:
+            logging.warning(
+                f"More than one regressor config found in {config_path}, using the first one"
+            )
+        self.regressor_settings = regressor_settings[list(regressor_settings.keys())[0]]
+        if "kernel" in self.regressor_settings["regressor_params"]:
+            from sklearn.gaussian_process.kernels import ConstantKernel, Matern  # noqa: F401
+
+            # We could hard-code the kernels they have, maybe better than using eval
+            self.regressor_settings["regressor_params"]["kernel"] = eval(
+                self.regressor_settings["regressor_params"]["kernel"]
+            )
+        # We assume the YAML has the key regressor containing a regressor name
+        self.regressor = self.get_regressor_by_name(self.regressor_settings["regressor"])
+        if "regressor_params" in self.regressor_settings:  # Set params if any are given
+            self.regressor.set_params(**self.regressor_settings["regressor_params"])
+
+        # TODO we could make this configurable
+        self.descriptors = Mordred()
+        self.descriptor_subset = None
+
+        self.min_max_scaler = MinMaxScaler().set_output(transform="polars")
+        self.feature_preselector = Pipeline(
+            [
+                (
+                    "variance_threshold",
+                    VarianceThreshold(threshold=0.02).set_output(transform="polars"),
+                ),
+                # Feature selection based on variance threshold
+                (
+                    "custom_feature_selection",
+                    FunctionTransformer(
+                        func=self.remove_highly_correlated_features,
+                        validate=False,
+                        kw_args={"corr_method": "spearman", "cluster_threshold": 0.01},
+                    ).set_output(transform="polars"),
+                ),
+            ]
+        )
+
+    def get_regressor_by_name(self, regressor_string):
+        """
+        Load regressor function from a regressor name
+        :param regressor_string: name of regressor as defined in config file (function name with parentheses)
+        :return: Regressor object
+        """
+        # if regressor_string == 'RandomForestRegressor':
+        #     return RandomForestRegressor(random_state=self.random_state)
+        # elif regressor_string == 'GradientBoostingRegressor':
+        #     return GradientBoostingRegressor(random_state=self.random_state)
+        # elif regressor_string == 'AdaBoostRegressor':
+        #     return AdaBoostRegressor(random_state=self.random_state)
+        # elif regressor_string == 'MLPRegressor':
+        #     return MLPRegressor(random_state=self.random_state)
+        # elif regressor_string == 'SVR':
+        #     return SVR()
+        # elif regressor_string == 'KNeighborsRegressor':
+        #     return KNeighborsRegressor()
+        if regressor_string == "GaussianProcessRegressor":
+            return GaussianProcessRegressor(random_state=self.random_state)
+        # elif regressor_string == 'DecisionTreeRegressor':
+        #     return DecisionTreeRegressor(random_state=self.random_state)
+        # elif regressor_string == 'Ridge':
+        #     return Ridge(random_state=self.random_state)
+        # elif regressor_string == 'SGDRegressor':
+        #     return SGDRegressor(random_state=self.random_state)
+        # elif regressor_string == 'KernelRidge':
+        #     return KernelRidge()
+        # elif regressor_string == 'LinearRegression':
+        #     return LinearRegression()
+        # elif regressor_string == 'LSVR':
+        #     return SVR(kernel='linear')  # Linear Support Vector Regressor
+        else:
+            raise NotImplementedError(
+                f"No regressor type defined for regressor_string = {regressor_string}"
+            )
+
+    def train_model(self, train_data, preprocess=True):
+        """
+        Fit self.regressor and preprocessors. train_data is a pl.DataFrame
+        """
+        if preprocess:
+            # Compute the mean and std of half-lives per structure
+            train_data = self.preprocess_data(train_data)
+
+        # train_data structure:
+        # columns = [
+        #     "structure_id",
+        #     "smiles",
+        #     "dt50_log",
+        #     "dt50_bayesian_mean",
+        #     "dt50_bayesian_std",
+        # ] + self.descriptors.get_descriptor_names()
+
+        # only select descriptor features for feature preselector
+        df = train_data[self.descriptors.get_descriptor_names()]
+
+        # Remove columns having at least None, nan, inf, "" value
+        df = Pepper.keep_clean_columns(df)
+
+        # Scale and Remove highly correlated features as well as features having a low variance
+        x_train_normal = self.min_max_scaler.fit_transform(df)
+        x_train_normal = self.feature_preselector.fit_transform(x_train_normal)
+
+        # Store subset, as this is the input used for prediction
+        self.descriptor_subset = x_train_normal.columns
+
+        y_train = train_data["dt50_bayesian_mean"].to_numpy()
+        y_train_std = train_data["dt50_bayesian_std"].to_numpy()
+
+        self.regressor.set_params(alpha=y_train_std)
+        self.regressor.fit(x_train_normal, y_train)
+
+        return self, train_data
+
+    @staticmethod
+    def keep_clean_columns(df: pl.DataFrame) -> pl.DataFrame:
+        """
+        Filters out columns from the DataFrame that contain null values, NaN, or infinite values.
+
+        This static method takes a DataFrame as input and evaluates each of its columns to determine
+        if the column contains invalid values. Columns that have null values, NaN, or infinite values
+        are excluded from the resulting DataFrame. The method is especially useful for cleaning up a
+        dataset by keeping only the valid columns.
+
+        Parameters:
+        df (polars.DataFrame): The input DataFrame to be cleaned.
+
+        Returns:
+        polars.DataFrame: A DataFrame containing only columns without null, NaN, or infinite values.
+        """
+        valid_cols = []
+
+        for col in df.columns:
+            s = df[col]
+
+            # Check nulls
+            has_null = s.null_count() > 0
+
+            # Check NaN and inf only for numeric columns
+            if s.dtype.is_numeric():
+                has_nan = s.is_nan().any()
+                has_inf = s.is_infinite().any()
+            else:
+                has_nan = False
+                has_inf = False
+
+            if not (has_null or has_nan or has_inf):
+                valid_cols.append(col)
+
+        return df.select(valid_cols)
+
+    def preprocess_data(self, dataset):
+        groups = [group for group in dataset.group_by("structure_id")]
+
+        # Unless explicitly set compute everything serial
+        if os.environ.get("N_PEPPER_THREADS", 1) > 1:
+            results = Parallel(n_jobs=os.environ["N_PEPPER_THREADS"])(
+                delayed(compute_bayes_per_group)(group[1])
+                for group in dataset.group_by("structure_id")
+            )
+        else:
+            results = []
+            for g in groups:
+                results.append(compute_bayes_per_group(g[1]))
+
+        bayes_stats = pl.concat(results, how="vertical")
+        dataset = dataset.join(bayes_stats, on="structure_id", how="left")
+
+        # Remove duplicates after calculating mean, std
+        dataset = dataset.unique(subset="structure_id")
+
+        # Calculate and normalise features, make a "desc" column with the features
+        dataset = dataset.with_columns(
+            pl.col("smiles")
+            .map_elements(
+                self.descriptors.get_molecule_descriptors, return_dtype=pl.List(pl.Float64)
+            )
+            .alias("desc")
+        )
+
+        # If a SMILES fails to get desc it is removed
+        dataset = dataset.filter(pl.col("desc").is_not_null() & (pl.col("desc").list.len() > 0))
+
+        # Flatten the features into the dataset
+        dataset = dataset.with_columns(
+            pl.col("desc").list.to_struct(fields=self.descriptors.get_descriptor_names())
+        ).unnest("desc")
+
+        return dataset
+
+    def predict_batch(self, batch: List[str], is_smiles: bool = True) -> List[List[float | None]]:
+        if is_smiles:
+            rows = [self.descriptors.get_molecule_descriptors(smiles) for smiles in batch]
+        else:
+            rows = batch
+
+        # Create Dataframe with all descriptors
+        initial_desc_rows_df = pl.DataFrame(
+            data=rows, schema=self.descriptors.get_descriptor_names(), orient="row"
+        )
+
+        # Before checking for invalid values per row, select only required columns
+        initial_desc_rows_df = initial_desc_rows_df.select(
+            list(self.min_max_scaler.feature_names_in_)
+        )
+
+        to_pad = []
+        adjusted_rows = []
+        for i, row in enumerate(initial_desc_rows_df.rows()):
+            # neither infs nor nans are found -> rows seems to be valid input
+            if row and not any(math.isinf(x) for x in row) and not any(math.isnan(x) for x in row):
+                adjusted_rows.append(row)
+            else:
+                to_pad.append(i)
+
+        if adjusted_rows:
+            desc_rows_df = pl.DataFrame(
+                data=adjusted_rows, schema=list(self.min_max_scaler.feature_names_in_), orient="row"
+            )
+            x_normal = self.min_max_scaler.transform(desc_rows_df)
+            x_normal = x_normal[self.descriptor_subset]
+
+            res = self.regressor.predict(x_normal, return_std=True)
+
+            # Convert to lists
+            res = [list(res[0]), list(res[1])]
+
+            # If we had rows containing bad input (inf, nan) insert Nones at the correct position
+            if to_pad:
+                for i in to_pad:
+                    res[0].insert(i, None)
+                    res[1].insert(i, None)
+
+            return res
+
+        else:
+            return [[None] * len(batch), [None] * len(batch)]
+
+    @staticmethod
+    def remove_highly_correlated_features(
+        X_train,
+        corr_method: str = "spearman",
+        cluster_threshold: float = 0.01,
+        ignore=False,
+    ):
+        if ignore:
+            return X_train
+            # pass
+        else:
+            # Using spearmanr from scipy to achieve pandas.corr in polars
+            corr = spearmanr(X_train, axis=0).statistic
+
+            # Ensure the correlation matrix is symmetric
+            corr = (corr + corr.T) / 2
+            np.fill_diagonal(corr, 1)
+            corr = np.nan_to_num(corr)
+
+            # code from https://scikit-learn.org/stable/auto_examples/inspection/
+            # plot_permutation_importance_multicollinear.html
+            # We convert the correlation matrix to a distance matrix before performing
+            # hierarchical clustering using Ward's linkage.
+            distance_matrix = 1 - np.abs(corr)
+            dist_linkage = hierarchy.ward(squareform(distance_matrix))
+
+            cluster_ids = hierarchy.fcluster(dist_linkage, cluster_threshold, criterion="distance")
+            cluster_id_to_feature_ids = defaultdict(list)
+
+            for idx, cluster_id in enumerate(cluster_ids):
+                cluster_id_to_feature_ids[cluster_id].append(idx)
+
+            my_selected_features = [v[0] for v in cluster_id_to_feature_ids.values()]
+            X_train_sel = X_train[:, my_selected_features]
+
+            return X_train_sel
+
+    def save_model(self, path):
+        with open(path, "wb") as save_file:
+            pickle.dump(self, save_file, protocol=5)
+
+    @staticmethod
+    def load_model(path) -> "Pepper":
+        with open(path, "rb") as load_file:
+            return pickle.load(load_file)
+
+
+def compute_bayes_per_group(group):
+    """Get mean and std using bayesian"""
+    mean, std = Bayesian(group["dt50_log"]).get_posterior_distribution()
+    return pl.DataFrame(
+        {
+            "structure_id": [group["structure_id"][0]],
+            "dt50_bayesian_mean": [mean],
+            "dt50_bayesian_std": [std],
+        }
+    )

pnpm-lock.yaml

@@ -9,29 +9,29 @@ importers:
   .:
     devDependencies:
       '@tailwindcss/cli':
-        specifier: ^4.1.16
-        version: 4.1.16
+        specifier: ^4.1.18
+        version: 4.1.18
       '@tailwindcss/postcss':
-        specifier: ^4.1.16
-        version: 4.1.16
+        specifier: ^4.1.18
+        version: 4.1.18
       daisyui:
-        specifier: ^5.4.3
-        version: 5.4.3
+        specifier: ^5.5.14
+        version: 5.5.14
       postcss:
         specifier: ^8.5.6
         version: 8.5.6
       prettier:
-        specifier: ^3.6.2
-        version: 3.6.2
+        specifier: ^3.7.4
+        version: 3.7.4
       prettier-plugin-jinja-template:
         specifier: ^2.1.0
-        version: 2.1.0(prettier@3.6.2)
+        version: 2.1.0(prettier@3.7.4)
       prettier-plugin-tailwindcss:
-        specifier: ^0.7.1
-        version: 0.7.1(prettier@3.6.2)
+        specifier: ^0.7.2
+        version: 0.7.2(prettier@3.7.4)
       tailwindcss:
-        specifier: ^4.1.16
-        version: 4.1.16
+        specifier: ^4.1.18
+        version: 4.1.18
 
 packages:
 
@@ -137,69 +137,69 @@ packages:
     resolution: {integrity: sha512-dfUnCxiN9H4ap84DvD2ubjw+3vUNpstxa0TneY/Paat8a3R4uQZDLSvWjmznAY/DoahqTHl9V46HF/Zs3F29pg==}
     engines: {node: '>= 10.0.0'}
 
-  '@tailwindcss/cli@4.1.16':
-    resolution: {integrity: sha512-dsnANPrh2ZooHyZ/8uJhc9ecpcYtufToc21NY09NS9vF16rxPCjJ8dP7TUAtPqlUJTHSmRkN2hCdoYQIlgh4fw==}
+  '@tailwindcss/cli@4.1.18':
+    resolution: {integrity: sha512-sMZ+lZbDyxwjD2E0L7oRUjJ01Ffjtme5OtjvvnC+cV4CEDcbqzbp25TCpxHj6kWLU9+DlqJOiNgSOgctC2aZmg==}
     hasBin: true
 
-  '@tailwindcss/node@4.1.16':
-    resolution: {integrity: sha512-BX5iaSsloNuvKNHRN3k2RcCuTEgASTo77mofW0vmeHkfrDWaoFAFvNHpEgtu0eqyypcyiBkDWzSMxJhp3AUVcw==}
+  '@tailwindcss/node@4.1.18':
+    resolution: {integrity: sha512-DoR7U1P7iYhw16qJ49fgXUlry1t4CpXeErJHnQ44JgTSKMaZUdf17cfn5mHchfJ4KRBZRFA/Coo+MUF5+gOaCQ==}
 
-  '@tailwindcss/oxide-android-arm64@4.1.16':
-    resolution: {integrity: sha512-8+ctzkjHgwDJ5caq9IqRSgsP70xhdhJvm+oueS/yhD5ixLhqTw9fSL1OurzMUhBwE5zK26FXLCz2f/RtkISqHA==}
+  '@tailwindcss/oxide-android-arm64@4.1.18':
+    resolution: {integrity: sha512-dJHz7+Ugr9U/diKJA0W6N/6/cjI+ZTAoxPf9Iz9BFRF2GzEX8IvXxFIi/dZBloVJX/MZGvRuFA9rqwdiIEZQ0Q==}
     engines: {node: '>= 10'}
     cpu: [arm64]
     os: [android]
 
-  '@tailwindcss/oxide-darwin-arm64@4.1.16':
-    resolution: {integrity: sha512-C3oZy5042v2FOALBZtY0JTDnGNdS6w7DxL/odvSny17ORUnaRKhyTse8xYi3yKGyfnTUOdavRCdmc8QqJYwFKA==}
+  '@tailwindcss/oxide-darwin-arm64@4.1.18':
+    resolution: {integrity: sha512-Gc2q4Qhs660bhjyBSKgq6BYvwDz4G+BuyJ5H1xfhmDR3D8HnHCmT/BSkvSL0vQLy/nkMLY20PQ2OoYMO15Jd0A==}
     engines: {node: '>= 10'}
     cpu: [arm64]
     os: [darwin]
 
-  '@tailwindcss/oxide-darwin-x64@4.1.16':
-    resolution: {integrity: sha512-vjrl/1Ub9+JwU6BP0emgipGjowzYZMjbWCDqwA2Z4vCa+HBSpP4v6U2ddejcHsolsYxwL5r4bPNoamlV0xDdLg==}
+  '@tailwindcss/oxide-darwin-x64@4.1.18':
+    resolution: {integrity: sha512-FL5oxr2xQsFrc3X9o1fjHKBYBMD1QZNyc1Xzw/h5Qu4XnEBi3dZn96HcHm41c/euGV+GRiXFfh2hUCyKi/e+yw==}
     engines: {node: '>= 10'}
     cpu: [x64]
     os: [darwin]
 
-  '@tailwindcss/oxide-freebsd-x64@4.1.16':
-    resolution: {integrity: sha512-TSMpPYpQLm+aR1wW5rKuUuEruc/oOX3C7H0BTnPDn7W/eMw8W+MRMpiypKMkXZfwH8wqPIRKppuZoedTtNj2tg==}
+  '@tailwindcss/oxide-freebsd-x64@4.1.18':
+    resolution: {integrity: sha512-Fj+RHgu5bDodmV1dM9yAxlfJwkkWvLiRjbhuO2LEtwtlYlBgiAT4x/j5wQr1tC3SANAgD+0YcmWVrj8R9trVMA==}
     engines: {node: '>= 10'}
     cpu: [x64]
     os: [freebsd]
 
-  '@tailwindcss/oxide-linux-arm-gnueabihf@4.1.16':
-    resolution: {integrity: sha512-p0GGfRg/w0sdsFKBjMYvvKIiKy/LNWLWgV/plR4lUgrsxFAoQBFrXkZ4C0w8IOXfslB9vHK/JGASWD2IefIpvw==}
+  '@tailwindcss/oxide-linux-arm-gnueabihf@4.1.18':
+    resolution: {integrity: sha512-Fp+Wzk/Ws4dZn+LV2Nqx3IilnhH51YZoRaYHQsVq3RQvEl+71VGKFpkfHrLM/Li+kt5c0DJe/bHXK1eHgDmdiA==}
     engines: {node: '>= 10'}
     cpu: [arm]
     os: [linux]
 
-  '@tailwindcss/oxide-linux-arm64-gnu@4.1.16':
-    resolution: {integrity: sha512-DoixyMmTNO19rwRPdqviTrG1rYzpxgyYJl8RgQvdAQUzxC1ToLRqtNJpU/ATURSKgIg6uerPw2feW0aS8SNr/w==}
+  '@tailwindcss/oxide-linux-arm64-gnu@4.1.18':
+    resolution: {integrity: sha512-S0n3jboLysNbh55Vrt7pk9wgpyTTPD0fdQeh7wQfMqLPM/Hrxi+dVsLsPrycQjGKEQk85Kgbx+6+QnYNiHalnw==}
     engines: {node: '>= 10'}
     cpu: [arm64]
     os: [linux]
 
-  '@tailwindcss/oxide-linux-arm64-musl@4.1.16':
-    resolution: {integrity: sha512-H81UXMa9hJhWhaAUca6bU2wm5RRFpuHImrwXBUvPbYb+3jo32I9VIwpOX6hms0fPmA6f2pGVlybO6qU8pF4fzQ==}
+  '@tailwindcss/oxide-linux-arm64-musl@4.1.18':
+    resolution: {integrity: sha512-1px92582HkPQlaaCkdRcio71p8bc8i/ap5807tPRDK/uw953cauQBT8c5tVGkOwrHMfc2Yh6UuxaH4vtTjGvHg==}
     engines: {node: '>= 10'}
     cpu: [arm64]
     os: [linux]
 
-  '@tailwindcss/oxide-linux-x64-gnu@4.1.16':
-    resolution: {integrity: sha512-ZGHQxDtFC2/ruo7t99Qo2TTIvOERULPl5l0K1g0oK6b5PGqjYMga+FcY1wIUnrUxY56h28FxybtDEla+ICOyew==}
+  '@tailwindcss/oxide-linux-x64-gnu@4.1.18':
+    resolution: {integrity: sha512-v3gyT0ivkfBLoZGF9LyHmts0Isc8jHZyVcbzio6Wpzifg/+5ZJpDiRiUhDLkcr7f/r38SWNe7ucxmGW3j3Kb/g==}
     engines: {node: '>= 10'}
     cpu: [x64]
     os: [linux]
 
-  '@tailwindcss/oxide-linux-x64-musl@4.1.16':
-    resolution: {integrity: sha512-Oi1tAaa0rcKf1Og9MzKeINZzMLPbhxvm7rno5/zuP1WYmpiG0bEHq4AcRUiG2165/WUzvxkW4XDYCscZWbTLZw==}
+  '@tailwindcss/oxide-linux-x64-musl@4.1.18':
+    resolution: {integrity: sha512-bhJ2y2OQNlcRwwgOAGMY0xTFStt4/wyU6pvI6LSuZpRgKQwxTec0/3Scu91O8ir7qCR3AuepQKLU/kX99FouqQ==}
     engines: {node: '>= 10'}
     cpu: [x64]
     os: [linux]
 
-  '@tailwindcss/oxide-wasm32-wasi@4.1.16':
-    resolution: {integrity: sha512-B01u/b8LteGRwucIBmCQ07FVXLzImWESAIMcUU6nvFt/tYsQ6IHz8DmZ5KtvmwxD+iTYBtM1xwoGXswnlu9v0Q==}
+  '@tailwindcss/oxide-wasm32-wasi@4.1.18':
+    resolution: {integrity: sha512-LffYTvPjODiP6PT16oNeUQJzNVyJl1cjIebq/rWWBF+3eDst5JGEFSc5cWxyRCJ0Mxl+KyIkqRxk1XPEs9x8TA==}
     engines: {node: '>=14.0.0'}
     cpu: [wasm32]
     bundledDependencies:
@@ -210,31 +210,31 @@ packages:
       - '@emnapi/wasi-threads'
       - tslib
 
-  '@tailwindcss/oxide-win32-arm64-msvc@4.1.16':
-    resolution: {integrity: sha512-zX+Q8sSkGj6HKRTMJXuPvOcP8XfYON24zJBRPlszcH1Np7xuHXhWn8qfFjIujVzvH3BHU+16jBXwgpl20i+v9A==}
+  '@tailwindcss/oxide-win32-arm64-msvc@4.1.18':
+    resolution: {integrity: sha512-HjSA7mr9HmC8fu6bdsZvZ+dhjyGCLdotjVOgLA2vEqxEBZaQo9YTX4kwgEvPCpRh8o4uWc4J/wEoFzhEmjvPbA==}
     engines: {node: '>= 10'}
     cpu: [arm64]
     os: [win32]
 
-  '@tailwindcss/oxide-win32-x64-msvc@4.1.16':
-    resolution: {integrity: sha512-m5dDFJUEejbFqP+UXVstd4W/wnxA4F61q8SoL+mqTypId2T2ZpuxosNSgowiCnLp2+Z+rivdU0AqpfgiD7yCBg==}
+  '@tailwindcss/oxide-win32-x64-msvc@4.1.18':
+    resolution: {integrity: sha512-bJWbyYpUlqamC8dpR7pfjA0I7vdF6t5VpUGMWRkXVE3AXgIZjYUYAK7II1GNaxR8J1SSrSrppRar8G++JekE3Q==}
     engines: {node: '>= 10'}
     cpu: [x64]
     os: [win32]
 
-  '@tailwindcss/oxide@4.1.16':
-    resolution: {integrity: sha512-2OSv52FRuhdlgyOQqgtQHuCgXnS8nFSYRp2tJ+4WZXKgTxqPy7SMSls8c3mPT5pkZ17SBToGM5LHEJBO7miEdg==}
+  '@tailwindcss/oxide@4.1.18':
+    resolution: {integrity: sha512-EgCR5tTS5bUSKQgzeMClT6iCY3ToqE1y+ZB0AKldj809QXk1Y+3jB0upOYZrn9aGIzPtUsP7sX4QQ4XtjBB95A==}
     engines: {node: '>= 10'}
 
-  '@tailwindcss/postcss@4.1.16':
-    resolution: {integrity: sha512-Qn3SFGPXYQMKR/UtqS+dqvPrzEeBZHrFA92maT4zijCVggdsXnDBMsPFJo1eArX3J+O+Gi+8pV4PkqjLCNBk3A==}
+  '@tailwindcss/postcss@4.1.18':
+    resolution: {integrity: sha512-Ce0GFnzAOuPyfV5SxjXGn0CubwGcuDB0zcdaPuCSzAa/2vII24JTkH+I6jcbXLb1ctjZMZZI6OjDaLPJQL1S0g==}
 
   braces@3.0.3:
     resolution: {integrity: sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==}
     engines: {node: '>=8'}
 
-  daisyui@5.4.3:
-    resolution: {integrity: sha512-dfDCJnN4utErGoWfElgdEE252FtfHV9Mxj5Dq1+JzUq3nVkluWdF3JYykP0Xy/y/yArnPXYztO1tLNCow3kjmg==}
+  daisyui@5.5.14:
+    resolution: {integrity: sha512-L47rvw7I7hK68TA97VB8Ee0woHew+/ohR6Lx6Ah/krfISOqcG4My7poNpX5Mo5/ytMxiR40fEaz6njzDi7cuSg==}
 
   detect-libc@1.0.3:
     resolution: {integrity: sha512-pGjwhsmsp4kL2RTz08wcOlGN83otlqHeD/Z5T8GXZB+/YcpQ/dgo+lbU8ZsGxV0HIvqqxo9l7mqYwyYMD9bKDg==}
@@ -245,8 +245,8 @@ packages:
     resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==}
     engines: {node: '>=8'}
 
-  enhanced-resolve@5.18.3:
-    resolution: {integrity: sha512-d4lC8xfavMeBjzGr2vECC3fsGXziXZQyJxD868h2M/mBI3PwAuODxAkLkq5HYuvrPYcUtiLzsTo8U3PgX3Ocww==}
+  enhanced-resolve@5.18.4:
+    resolution: {integrity: sha512-LgQMM4WXU3QI+SYgEc2liRgznaD5ojbmY3sb8LxyguVkIg5FxdpTkvk72te2R38/TGKxH634oLxXRGY6d7AP+Q==}
     engines: {node: '>=10.13.0'}
 
   fill-range@7.1.1:
@@ -377,8 +377,8 @@ packages:
     peerDependencies:
       prettier: ^3.0.0
 
-  prettier-plugin-tailwindcss@0.7.1:
-    resolution: {integrity: sha512-Bzv1LZcuiR1Sk02iJTS1QzlFNp/o5l2p3xkopwOrbPmtMeh3fK9rVW5M3neBQzHq+kGKj/4LGQMTNcTH4NGPtQ==}
+  prettier-plugin-tailwindcss@0.7.2:
+    resolution: {integrity: sha512-LkphyK3Fw+q2HdMOoiEHWf93fNtYJwfamoKPl7UwtjFQdei/iIBoX11G6j706FzN3ymX9mPVi97qIY8328vdnA==}
     engines: {node: '>=20.19'}
     peerDependencies:
       '@ianvs/prettier-plugin-sort-imports': '*'
@@ -432,8 +432,8 @@ packages:
       prettier-plugin-svelte:
         optional: true
 
-  prettier@3.6.2:
-    resolution: {integrity: sha512-I7AIg5boAr5R0FFtJ6rCfD+LFsWHp81dolrFD8S79U9tb8Az2nGrJncnMSnys+bpQJfRUzqs9hnA81OAA3hCuQ==}
+  prettier@3.7.4:
+    resolution: {integrity: sha512-v6UNi1+3hSlVvv8fSaoUbggEM5VErKmmpGA7Pl3HF8V6uKY7rvClBOJlH6yNwQtfTueNkGVpOv/mtWL9L4bgRA==}
     engines: {node: '>=14'}
     hasBin: true
 
@@ -441,8 +441,8 @@ packages:
     resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==}
     engines: {node: '>=0.10.0'}
 
-  tailwindcss@4.1.16:
-    resolution: {integrity: sha512-pONL5awpaQX4LN5eiv7moSiSPd/DLDzKVRJz8Q9PgzmAdd1R4307GQS2ZpfiN7ZmekdQrfhZZiSE5jkLR4WNaA==}
+  tailwindcss@4.1.18:
+    resolution: {integrity: sha512-4+Z+0yiYyEtUVCScyfHCxOYP06L5Ne+JiHhY2IjR2KWMIWhJOYZKLSGZaP5HkZ8+bY0cxfzwDE5uOmzFXyIwxw==}
 
   tapable@2.3.0:
     resolution: {integrity: sha512-g9ljZiwki/LfxmQADO3dEY1CbpmXT5Hm2fJ+QaGKwSXUylMybePR7/67YW7jOrrvjEgL1Fmz5kzyAjWVWLlucg==}
@@ -535,96 +535,96 @@ snapshots:
       '@parcel/watcher-win32-ia32': 2.5.1
       '@parcel/watcher-win32-x64': 2.5.1
 
-  '@tailwindcss/cli@4.1.16':
+  '@tailwindcss/cli@4.1.18':
     dependencies:
       '@parcel/watcher': 2.5.1
-      '@tailwindcss/node': 4.1.16
-      '@tailwindcss/oxide': 4.1.16
-      enhanced-resolve: 5.18.3
+      '@tailwindcss/node': 4.1.18
+      '@tailwindcss/oxide': 4.1.18
+      enhanced-resolve: 5.18.4
       mri: 1.2.0
       picocolors: 1.1.1
-      tailwindcss: 4.1.16
+      tailwindcss: 4.1.18
 
-  '@tailwindcss/node@4.1.16':
+  '@tailwindcss/node@4.1.18':
     dependencies:
       '@jridgewell/remapping': 2.3.5
-      enhanced-resolve: 5.18.3
+      enhanced-resolve: 5.18.4
       jiti: 2.6.1
       lightningcss: 1.30.2
       magic-string: 0.30.21
       source-map-js: 1.2.1
-      tailwindcss: 4.1.16
+      tailwindcss: 4.1.18
 
-  '@tailwindcss/oxide-android-arm64@4.1.16':
+  '@tailwindcss/oxide-android-arm64@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-darwin-arm64@4.1.16':
+  '@tailwindcss/oxide-darwin-arm64@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-darwin-x64@4.1.16':
+  '@tailwindcss/oxide-darwin-x64@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-freebsd-x64@4.1.16':
+  '@tailwindcss/oxide-freebsd-x64@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-linux-arm-gnueabihf@4.1.16':
+  '@tailwindcss/oxide-linux-arm-gnueabihf@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-linux-arm64-gnu@4.1.16':
+  '@tailwindcss/oxide-linux-arm64-gnu@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-linux-arm64-musl@4.1.16':
+  '@tailwindcss/oxide-linux-arm64-musl@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-linux-x64-gnu@4.1.16':
+  '@tailwindcss/oxide-linux-x64-gnu@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-linux-x64-musl@4.1.16':
+  '@tailwindcss/oxide-linux-x64-musl@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-wasm32-wasi@4.1.16':
+  '@tailwindcss/oxide-wasm32-wasi@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-win32-arm64-msvc@4.1.16':
+  '@tailwindcss/oxide-win32-arm64-msvc@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide-win32-x64-msvc@4.1.16':
+  '@tailwindcss/oxide-win32-x64-msvc@4.1.18':
     optional: true
 
-  '@tailwindcss/oxide@4.1.16':
+  '@tailwindcss/oxide@4.1.18':
     optionalDependencies:
-      '@tailwindcss/oxide-android-arm64': 4.1.16
-      '@tailwindcss/oxide-darwin-arm64': 4.1.16
-      '@tailwindcss/oxide-darwin-x64': 4.1.16
-      '@tailwindcss/oxide-freebsd-x64': 4.1.16
-      '@tailwindcss/oxide-linux-arm-gnueabihf': 4.1.16
-      '@tailwindcss/oxide-linux-arm64-gnu': 4.1.16
-      '@tailwindcss/oxide-linux-arm64-musl': 4.1.16
-      '@tailwindcss/oxide-linux-x64-gnu': 4.1.16
-      '@tailwindcss/oxide-linux-x64-musl': 4.1.16
-      '@tailwindcss/oxide-wasm32-wasi': 4.1.16
-      '@tailwindcss/oxide-win32-arm64-msvc': 4.1.16
-      '@tailwindcss/oxide-win32-x64-msvc': 4.1.16
+      '@tailwindcss/oxide-android-arm64': 4.1.18
+      '@tailwindcss/oxide-darwin-arm64': 4.1.18
+      '@tailwindcss/oxide-darwin-x64': 4.1.18
+      '@tailwindcss/oxide-freebsd-x64': 4.1.18
+      '@tailwindcss/oxide-linux-arm-gnueabihf': 4.1.18
+      '@tailwindcss/oxide-linux-arm64-gnu': 4.1.18
+      '@tailwindcss/oxide-linux-arm64-musl': 4.1.18
+      '@tailwindcss/oxide-linux-x64-gnu': 4.1.18
+      '@tailwindcss/oxide-linux-x64-musl': 4.1.18
+      '@tailwindcss/oxide-wasm32-wasi': 4.1.18
+      '@tailwindcss/oxide-win32-arm64-msvc': 4.1.18
+      '@tailwindcss/oxide-win32-x64-msvc': 4.1.18
 
-  '@tailwindcss/postcss@4.1.16':
+  '@tailwindcss/postcss@4.1.18':
     dependencies:
       '@alloc/quick-lru': 5.2.0
-      '@tailwindcss/node': 4.1.16
-      '@tailwindcss/oxide': 4.1.16
+      '@tailwindcss/node': 4.1.18
+      '@tailwindcss/oxide': 4.1.18
       postcss: 8.5.6
-      tailwindcss: 4.1.16
+      tailwindcss: 4.1.18
 
   braces@3.0.3:
     dependencies:
       fill-range: 7.1.1
 
-  daisyui@5.4.3: {}
+  daisyui@5.5.14: {}
 
   detect-libc@1.0.3: {}
 
   detect-libc@2.1.2: {}
 
-  enhanced-resolve@5.18.3:
+  enhanced-resolve@5.18.4:
     dependencies:
       graceful-fs: 4.2.11
       tapable: 2.3.0
@@ -719,19 +719,19 @@ snapshots:
       picocolors: 1.1.1
       source-map-js: 1.2.1
 
-  prettier-plugin-jinja-template@2.1.0(prettier@3.6.2):
+  prettier-plugin-jinja-template@2.1.0(prettier@3.7.4):
     dependencies:
-      prettier: 3.6.2
+      prettier: 3.7.4
 
-  prettier-plugin-tailwindcss@0.7.1(prettier@3.6.2):
+  prettier-plugin-tailwindcss@0.7.2(prettier@3.7.4):
     dependencies:
-      prettier: 3.6.2
+      prettier: 3.7.4
 
-  prettier@3.6.2: {}
+  prettier@3.7.4: {}
 
   source-map-js@1.2.1: {}
 
-  tailwindcss@4.1.16: {}
+  tailwindcss@4.1.18: {}
 
   tapable@2.3.0: {}
 

pnpm-workspace.yaml

@@ -0,0 +1,3 @@
+onlyBuiltDependencies:
+  - '@parcel/watcher'
+  - '@tailwindcss/oxide'

pyproject.toml

@@ -19,6 +19,7 @@ dependencies = [
     "envipy-plugins",
     "epam-indigo>=1.30.1",
     "gunicorn>=23.0.0",
+    "jsonref>=1.1.0",
     "networkx>=3.4.2",
     "psycopg2-binary>=2.9.10",
     "python-dotenv>=1.1.0",
@@ -35,7 +36,7 @@ dependencies = [
 [tool.uv.sources]
 enviformer = { git = "ssh://git@git.envipath.com/enviPath/enviformer.git", rev = "v0.1.4" }
 envipy-plugins = { git = "ssh://git@git.envipath.com/enviPath/enviPy-plugins.git", rev = "v0.1.0" }
-envipy-additional-information = { git = "ssh://git@git.envipath.com/enviPath/enviPy-additional-information.git", rev = "v0.1.7" }
+envipy-additional-information = { git = "ssh://git@git.envipath.com/enviPath/enviPy-additional-information.git", branch = "develop" }
 envipy-ambit = { git = "ssh://git@git.envipath.com/enviPath/enviPy-ambit.git" }
 
 [project.optional-dependencies]
@@ -50,7 +51,13 @@ dev = [
     "pytest-django>=4.11.1",
     "pytest-cov>=7.0.0",
 ]
-
+pepper-plugin = [
+    "matplotlib>=3.10.8",
+    "pyyaml>=6.0.3",
+    "emcee>=3.1.6",
+    "mordredcommunity==2.0.7",
+    "padelpy" # Remove once we're certain we'll go with mordred
+]
 [tool.ruff]
 line-length = 100
 
@@ -85,8 +92,15 @@ build = { sequence = [
 ], help = "Build frontend assets and collect static files" }
 
 # Database tasks
-db-up = { cmd = "docker compose -f docker-compose.dev.yml up -d", help = "Start PostgreSQL database using Docker Compose" }
-db-down = { cmd = "docker compose -f docker-compose.dev.yml down", help = "Stop PostgreSQL database" }
+db-up = { cmd = "docker compose -p envipath -f docker-compose.dev.yml up -d", help = "Start PostgreSQL database using Docker Compose" }
+db-down = { cmd = "docker compose -p envipath -f docker-compose.dev.yml down", help = "Stop PostgreSQL database" }
+
+# Celery tasks
+celery = { cmd = "celery -A envipath worker -l INFO -Q predict,model,background", help = "Start Celery worker for async task processing" }
+celery-dev = { sequence = [
+    "db-up",
+    "celery",
+], help = "Start database and Celery worker" }
 
 # Frontend tasks
 js-deps = { cmd = "uv run python scripts/pnpm_wrapper.py install", help = "Install frontend dependencies" }

scripts/dev_server.py

@@ -11,6 +11,8 @@ import signal
 import subprocess
 import sys
 import time
+import os
+import dotenv
 
 
 def find_pnpm():
@@ -65,6 +67,7 @@ class DevServerManager:
                     bufsize=1,
                 )
             self.processes.append((process, description))
+            print(" ".join(command))
             print(f"✓ Started {description} (PID: {process.pid})")
             return process
         except Exception as e:
@@ -146,6 +149,7 @@ class DevServerManager:
 
 def main():
     """Main entry point."""
+    dotenv.load_dotenv()
     manager = DevServerManager()
     manager.register_cleanup()
 
@@ -174,9 +178,10 @@ def main():
     time.sleep(1)
 
     # Start Django dev server
+    port = os.environ.get("DJANGO_PORT", "8000")
     django_process = manager.start_process(
-        ["uv", "run", "python", "manage.py", "runserver"],
-        "Django server",
+        ["uv", "run", "python", "manage.py", "runserver", f"0:{port}"],
+        f"Django server on port {port}",
         shell=False,
     )
 

static/js/alpine/components/schema-form.js

@@ -0,0 +1,378 @@
+/**
+ * Alpine.js Schema Renderer Component
+ *
+ * Renders forms dynamically from JSON Schema with RJSF format support.
+ * Supports uiSchema for widget hints, labels, help text, and field ordering.
+ *
+ * Usage:
+ * <div x-data="schemaRenderer({
+ *     rjsf: { schema: {...}, uiSchema: {...}, formData: {...}, groups: [...] },
+ *     data: { interval: { start: 20, end: 25 } },
+ *     mode: 'view', // 'view' | 'edit'
+ *     endpoint: '/api/v1/scenario/{uuid}/information/temperature/'
+ * })">
+ */
+document.addEventListener("alpine:init", () => {
+  // Global validation error store with context scoping
+  Alpine.store('validationErrors', {
+    errors: {},
+
+    // Set errors for a specific context (UUID) or globally (no context)
+    setErrors(errors, context = null) {
+      if (context) {
+        // Namespace all field names with context prefix
+        const namespacedErrors = {};
+        Object.entries(errors).forEach(([field, messages]) => {
+          const key = `${context}.${field}`;
+          namespacedErrors[key] = messages;
+        });
+        // Merge into existing errors (preserves other contexts)
+        this.errors = { ...this.errors, ...namespacedErrors };
+      } else {
+        // No context - merge as-is for backward compatibility
+        this.errors = { ...this.errors, ...errors };
+      }
+    },
+
+    // Clear errors for a specific context or all errors
+    clearErrors(context = null) {
+      if (context) {
+        // Clear only errors for this context
+        const newErrors = {};
+        const prefix = `${context}.`;
+        Object.keys(this.errors).forEach(key => {
+          if (!key.startsWith(prefix)) {
+            newErrors[key] = this.errors[key];
+          }
+        });
+        this.errors = newErrors;
+      } else {
+        // Clear all errors
+        this.errors = {};
+      }
+    },
+
+    // Clear a specific field, optionally within a context
+    clearField(fieldName, context = null) {
+      const key = context ? `${context}.${fieldName}` : fieldName;
+      if (this.errors[key]) {
+        delete this.errors[key];
+        // Trigger reactivity by creating new object
+        this.errors = { ...this.errors };
+      }
+    },
+
+    // Check if a field has errors, optionally within a context
+    hasError(fieldName, context = null) {
+      const key = context ? `${context}.${fieldName}` : fieldName;
+      return Array.isArray(this.errors[key]) && this.errors[key].length > 0;
+    },
+
+    // Get errors for a field, optionally within a context
+    getErrors(fieldName, context = null) {
+      const key = context ? `${context}.${fieldName}` : fieldName;
+      return this.errors[key] || [];
+    }
+  });
+
+  Alpine.data("schemaRenderer", (options = {}) => ({
+    schema: null,
+    uiSchema: {},
+    data: {},
+    mode: options.mode || "view", // 'view' | 'edit'
+    endpoint: options.endpoint || "",
+    loading: false,
+    error: null,
+    context: options.context || null, // UUID for items, null for single forms
+    debugErrors:
+      options.debugErrors ??
+      (typeof window !== "undefined" &&
+        window.location?.search?.includes("debugErrors=1")),
+
+    async init() {
+      if (options.schemaUrl) {
+        try {
+          this.loading = true;
+          const res = await fetch(options.schemaUrl);
+          if (!res.ok) {
+            throw new Error(`Failed to load schema: ${res.statusText}`);
+          }
+          const rjsf = await res.json();
+
+          // RJSF format: {schema, uiSchema, formData, groups}
+          if (!rjsf.schema) {
+            throw new Error("Invalid RJSF format: missing schema property");
+          }
+
+          this.schema = rjsf.schema;
+          this.uiSchema = rjsf.uiSchema || {};
+          this.data = options.data
+            ? JSON.parse(JSON.stringify(options.data))
+            : rjsf.formData || {};
+        } catch (err) {
+          this.error = err.message;
+          console.error("Error loading schema:", err);
+        } finally {
+          this.loading = false;
+        }
+      } else if (options.rjsf) {
+        // Direct RJSF object passed
+        if (!options.rjsf.schema) {
+          throw new Error("Invalid RJSF format: missing schema property");
+        }
+
+        this.schema = options.rjsf.schema;
+        this.uiSchema = options.rjsf.uiSchema || {};
+        this.data = options.data
+          ? JSON.parse(JSON.stringify(options.data))
+          : options.rjsf.formData || {};
+      }
+
+      // Initialize data from formData or options
+      if (!this.data || Object.keys(this.data).length === 0) {
+        this.data = {};
+      }
+
+      // Ensure all schema fields are properly initialized
+      if (this.schema && this.schema.properties) {
+        for (const [key, propSchema] of Object.entries(
+          this.schema.properties,
+        )) {
+          const widget = this.getWidget(key, propSchema);
+
+          if (widget === "interval") {
+            // Ensure interval fields are objects with start/end
+            if (!this.data[key] || typeof this.data[key] !== "object") {
+              this.data[key] = { start: null, end: null };
+            } else {
+              // Ensure start and end exist
+              if (this.data[key].start === undefined)
+                this.data[key].start = null;
+              if (this.data[key].end === undefined) this.data[key].end = null;
+            }
+          } else if (widget === "timeseries-table") {
+            // Ensure timeseries fields are arrays
+            if (!this.data[key] || !Array.isArray(this.data[key])) {
+              this.data[key] = [];
+            }
+          } else if (this.data[key] === undefined) {
+            // ONLY initialize if truly undefined, not just falsy
+            // This preserves empty strings, null, 0, false as valid values
+            if (propSchema.type === "boolean") {
+              this.data[key] = false;
+            } else if (
+              propSchema.type === "number" ||
+              propSchema.type === "integer"
+            ) {
+              this.data[key] = null;
+            } else if (propSchema.enum) {
+              // For select fields, use null to show placeholder
+              this.data[key] = null;
+            } else {
+              this.data[key] = "";
+            }
+          }
+          // If data[key] exists (even if empty string or null), don't overwrite
+        }
+      }
+
+      // UX: Clear field errors when fields change (with context)
+      if (this.mode === "edit" && this.schema?.properties) {
+        Object.keys(this.schema.properties).forEach((key) => {
+          this.$watch(
+            `data.${key}`,
+            () => {
+              Alpine.store('validationErrors').clearField(key, this.context);
+            },
+            { deep: true },
+          );
+        });
+      }
+    },
+
+    getWidget(fieldName, fieldSchema) {
+      // Defensive check: ensure fieldSchema is provided
+      if (!fieldSchema) return "text";
+
+      try {
+        // Check uiSchema first (RJSF format)
+        if (
+          this.uiSchema &&
+          this.uiSchema[fieldName] &&
+          this.uiSchema[fieldName]["ui:widget"]
+        ) {
+          return this.uiSchema[fieldName]["ui:widget"];
+        }
+
+        // Check for interval type (object with start/end properties)
+        if (
+          fieldSchema.type === "object" &&
+          fieldSchema.properties &&
+          fieldSchema.properties.start &&
+          fieldSchema.properties.end
+        ) {
+          return "interval";
+        }
+
+        // Check for measurements array type (timeseries-table widget)
+        if (
+          fieldSchema.type === "array" &&
+          fieldSchema.items?.properties?.timestamp &&
+          fieldSchema.items?.properties?.value
+        ) {
+          return "timeseries-table";
+        }
+
+        // Infer from JSON Schema type
+        if (fieldSchema.enum) return "select";
+        if (fieldSchema.type === "number" || fieldSchema.type === "integer")
+          return "number";
+        if (fieldSchema.type === "boolean") return "checkbox";
+        return "text";
+      } catch (e) {
+        // Fallback to text widget if anything fails
+        console.warn("Error in getWidget:", e);
+        return "text";
+      }
+    },
+
+    getLabel(fieldName, fieldSchema) {
+      // Check uiSchema (RJSF format)
+      if (this.uiSchema[fieldName] && this.uiSchema[fieldName]["ui:label"]) {
+        return this.uiSchema[fieldName]["ui:label"];
+      }
+
+      // Default: format field name
+      return fieldName
+        .replace(/_/g, " ")
+        .replace(/\b\w/g, (c) => c.toUpperCase());
+    },
+
+    getFieldOrder() {
+      try {
+        // Get ordered list of field names based on ui:order
+        if (!this.schema || !this.schema.properties) return [];
+
+        // Only include fields that have UI configs
+        const fields = Object.keys(this.schema.properties).filter(
+          (fieldName) => this.uiSchema && this.uiSchema[fieldName],
+        );
+
+        // Sort by ui:order if available
+        return fields.sort((a, b) => {
+          const orderA = this.uiSchema[a]?.["ui:order"] || "999";
+          const orderB = this.uiSchema[b]?.["ui:order"] || "999";
+          return parseInt(orderA) - parseInt(orderB);
+        });
+      } catch (e) {
+        // Return empty array if anything fails to prevent errors
+        console.warn("Error in getFieldOrder:", e);
+        return [];
+      }
+    },
+
+    hasTimeseriesField() {
+      try {
+        // Check if any field in the schema is a timeseries-table widget
+        if (!this.schema || !this.schema.properties) {
+          return false;
+        }
+
+        return Object.keys(this.schema.properties).some((fieldName) => {
+          const fieldSchema = this.schema.properties[fieldName];
+          if (!fieldSchema) return false;
+          return this.getWidget(fieldName, fieldSchema) === "timeseries-table";
+        });
+      } catch (e) {
+        // Return false if anything fails to prevent errors
+        console.warn("Error in hasTimeseriesField:", e);
+        return false;
+      }
+    },
+
+    async submit() {
+      if (!this.endpoint) {
+        console.error("No endpoint specified for submission");
+        return;
+      }
+
+      this.loading = true;
+      this.error = null;
+
+      try {
+        const csrftoken =
+          document.querySelector("[name=csrf-token]")?.content || "";
+        const res = await fetch(this.endpoint, {
+          method: "POST",
+          headers: {
+            "Content-Type": "application/json",
+            "X-CSRFToken": csrftoken,
+          },
+          body: JSON.stringify(this.data),
+        });
+
+        if (!res.ok) {
+          let errorData;
+          try {
+            errorData = await res.json();
+          } catch {
+            errorData = { error: res.statusText };
+          }
+
+          // Handle validation errors (field-level)
+          Alpine.store('validationErrors').clearErrors();
+
+          // Try to parse structured error response
+          let parsedError = errorData;
+
+          // If error is a JSON string, parse it
+          if (
+            typeof errorData.error === "string" &&
+            errorData.error.startsWith("{")
+          ) {
+            parsedError = JSON.parse(errorData.error);
+          }
+
+          if (parsedError.detail && Array.isArray(parsedError.detail)) {
+            // Pydantic validation errors format: [{loc: ['field'], msg: '...', type: '...'}]
+            const fieldErrors = {};
+            for (const err of parsedError.detail) {
+              const field =
+                err.loc && err.loc.length > 0
+                  ? err.loc[err.loc.length - 1]
+                  : "root";
+              if (!fieldErrors[field]) {
+                fieldErrors[field] = [];
+              }
+              fieldErrors[field].push(
+                err.msg || err.message || "Validation error",
+              );
+            }
+            Alpine.store('validationErrors').setErrors(fieldErrors);
+            throw new Error(
+              "Validation failed. Please check the fields below.",
+            );
+          } else {
+            // General error
+            throw new Error(
+              parsedError.error ||
+                parsedError.detail ||
+                `Request failed: ${res.statusText}`,
+            );
+          }
+        }
+
+        // Clear errors on success
+        Alpine.store('validationErrors').clearErrors();
+
+        const result = await res.json();
+        return result;
+      } catch (err) {
+        this.error = err.message;
+        throw err;
+      } finally {
+        this.loading = false;
+      }
+    },
+  }));
+});

static/js/alpine/components/widgets.js

@@ -0,0 +1,472 @@
+/**
+ * Alpine.js Widget Components for Schema Forms
+ *
+ * Centralized widget component definitions for dynamic form rendering.
+ * Each widget receives explicit parameters instead of context object for better traceability.
+ */
+document.addEventListener("alpine:init", () => {
+  // Base widget factory with common functionality
+  const baseWidget = (
+    fieldName,
+    data,
+    schema,
+    uiSchema,
+    mode,
+    debugErrors,
+    context = null  // NEW: context for error namespacing
+  ) => ({
+    fieldName,
+    data,
+    schema,
+    uiSchema,
+    mode,
+    debugErrors,
+    context,  // Store context for use in templates
+
+    // Field schema access
+    get fieldSchema() {
+      return this.schema?.properties?.[this.fieldName] || {};
+    },
+
+    // Common metadata
+    get label() {
+      // Check uiSchema first (RJSF format)
+      if (this.uiSchema?.[this.fieldName]?.["ui:label"]) {
+        return this.uiSchema[this.fieldName]["ui:label"];
+      }
+      // Fall back to schema title
+      if (this.fieldSchema.title) {
+        return this.fieldSchema.title;
+      }
+      // Default: format field name
+      return this.fieldName
+        .replace(/_/g, " ")
+        .replace(/\b\w/g, (c) => c.toUpperCase());
+    },
+    get helpText() {
+      return this.fieldSchema.description || "";
+    },
+
+    // Field-level unit extraction from uiSchema (RJSF format)
+    get unit() {
+      return this.uiSchema?.[this.fieldName]?.["ui:unit"] || null;
+    },
+
+    // Mode checks
+    get isViewMode() {
+      return this.mode === "view";
+    },
+    get isEditMode() {
+      return this.mode === "edit";
+    },
+  });
+
+  // Text widget
+  Alpine.data(
+    "textWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get value() {
+        return this.data[this.fieldName] || "";
+      },
+      set value(v) {
+        this.data[this.fieldName] = v;
+      },
+    }),
+  );
+
+  // Textarea widget
+  Alpine.data(
+    "textareaWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get value() {
+        return this.data[this.fieldName] || "";
+      },
+      set value(v) {
+        this.data[this.fieldName] = v;
+      },
+    }),
+  );
+
+  // Number widget with unit support
+  Alpine.data(
+    "numberWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get value() {
+        return this.data[this.fieldName];
+      },
+      set value(v) {
+        this.data[this.fieldName] =
+          v === "" || v === null ? null : parseFloat(v);
+      },
+      get hasValue() {
+        return (
+          this.value !== null && this.value !== undefined && this.value !== ""
+        );
+      },
+      // Format value with unit for view mode
+      get displayValue() {
+        if (!this.hasValue) return "—";
+        return this.unit ? `${this.value} ${this.unit}` : String(this.value);
+      },
+    }),
+  );
+
+  // Select widget
+  Alpine.data(
+    "selectWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get value() {
+        return this.data[this.fieldName] || "";
+      },
+      set value(v) {
+        this.data[this.fieldName] = v;
+      },
+      get multiple() {
+        return !!(this.fieldSchema.items && this.fieldSchema.items.enum);
+
+      },
+      get options() {
+        if (this.fieldSchema.enum) {
+          return this.fieldSchema.enum;
+        } else if (this.fieldSchema.items && this.fieldSchema.items.enum) {
+          return this.fieldSchema.items.enum;
+        } else {
+          return [];
+        }
+      },
+    }),
+  );
+
+  // Checkbox widget
+  Alpine.data(
+    "checkboxWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get checked() {
+        return !!this.data[this.fieldName];
+      },
+      set checked(v) {
+        this.data[this.fieldName] = v;
+      },
+    }),
+  );
+
+  // Interval widget with unit support
+  Alpine.data(
+    "intervalWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get start() {
+        return this.data[this.fieldName]?.start ?? null;
+      },
+      set start(v) {
+        if (!this.data[this.fieldName]) this.data[this.fieldName] = {};
+        this.data[this.fieldName].start =
+          v === "" || v === null ? null : parseFloat(v);
+      },
+      get end() {
+        return this.data[this.fieldName]?.end ?? null;
+      },
+      set end(v) {
+        if (!this.data[this.fieldName]) this.data[this.fieldName] = {};
+        this.data[this.fieldName].end =
+          v === "" || v === null ? null : parseFloat(v);
+      },
+      // Format interval with unit for view mode
+      get displayValue() {
+        const s = this.start,
+          e = this.end;
+        const unitStr = this.unit ? ` ${this.unit}` : "";
+
+        if (s !== null && e !== null) return `${s} – ${e}${unitStr}`;
+        if (s !== null) return `≥ ${s}${unitStr}`;
+        if (e !== null) return `≤ ${e}${unitStr}`;
+        return "—";
+      },
+
+      get isSameValue() {
+        return this.start !== null && this.start === this.end;
+      },
+
+      // Validation: start must be <= end (client-side)
+      get hasValidationError() {
+        if (this.isViewMode) return false;
+        const s = this.start;
+        const e = this.end;
+        // Only validate if both values are provided
+        if (
+          s !== null &&
+          e !== null &&
+          typeof s === "number" &&
+          typeof e === "number"
+        ) {
+          return s > e;
+        }
+        return false;
+      },
+    }),
+  );
+
+  // PubMed link widget
+  Alpine.data(
+    "pubmedWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get value() {
+        return this.data[this.fieldName] || "";
+      },
+      set value(v) {
+        this.data[this.fieldName] = v;
+      },
+      get pubmedUrl() {
+        return this.value
+          ? `https://pubmed.ncbi.nlm.nih.gov/${this.value}`
+          : null;
+      },
+    }),
+  );
+
+  // Compound link widget
+  Alpine.data(
+    "compoundWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      get value() {
+        return this.data[this.fieldName] || "";
+      },
+      set value(v) {
+        this.data[this.fieldName] = v;
+      },
+    }),
+  );
+
+  // TimeSeries table widget
+  Alpine.data(
+    "timeseriesTableWidget",
+    (fieldName, data, schema, uiSchema, mode, debugErrors, context = null) => ({
+      ...baseWidget(
+        fieldName,
+        data,
+        schema,
+        uiSchema,
+        mode,
+        debugErrors,
+        context,
+      ),
+
+      chartInstance: null,
+
+      // Getter/setter for measurements array
+      get measurements() {
+        return this.data[this.fieldName] || [];
+      },
+      set measurements(v) {
+        this.data[this.fieldName] = v;
+      },
+
+      // Get description from sibling field
+      get description() {
+        return this.data?.description || "";
+      },
+
+      // Get method from sibling field
+      get method() {
+        return this.data?.method || "";
+      },
+
+      // Computed property for chart options
+      get chartOptions() {
+        return {
+          measurements: this.measurements,
+          xAxisLabel: this.data?.x_axis_label || "Time",
+          yAxisLabel: this.data?.y_axis_label || "Value",
+          xAxisUnit: this.data?.x_axis_unit || "",
+          yAxisUnit: this.data?.y_axis_unit || "",
+        };
+      },
+
+      // Add new measurement
+      addMeasurement() {
+        if (!this.data[this.fieldName]) {
+          this.data[this.fieldName] = [];
+        }
+        this.data[this.fieldName].push({
+          timestamp: null,
+          value: null,
+          error: null,
+          note: "",
+        });
+      },
+
+      // Remove measurement by index
+      removeMeasurement(index) {
+        if (
+          this.data[this.fieldName] &&
+          Array.isArray(this.data[this.fieldName])
+        ) {
+          this.data[this.fieldName].splice(index, 1);
+        }
+      },
+
+      // Update specific measurement field
+      updateMeasurement(index, field, value) {
+        if (this.data[this.fieldName] && this.data[this.fieldName][index]) {
+          if (field === "timestamp" || field === "value" || field === "error") {
+            // Parse all numeric fields (timestamp is days as float)
+            this.data[this.fieldName][index][field] =
+              value === "" || value === null ? null : parseFloat(value);
+          } else {
+            // Store other fields as-is
+            this.data[this.fieldName][index][field] = value;
+          }
+        }
+      },
+
+      // Format timestamp for display (timestamp is numeric days as float)
+      formatTimestamp(timestamp) {
+        return timestamp ?? "";
+      },
+
+      // Sort by timestamp (numeric days)
+      sortByTimestamp() {
+        if (
+          this.data[this.fieldName] &&
+          Array.isArray(this.data[this.fieldName])
+        ) {
+          this.data[this.fieldName].sort((a, b) => {
+            const tsA = a.timestamp ?? Infinity;
+            const tsB = b.timestamp ?? Infinity;
+            return tsA - tsB;
+          });
+        }
+      },
+
+      // Chart lifecycle methods (delegates to TimeSeriesChart utility)
+      initChart() {
+        if (!this.isViewMode || !window.Chart || !window.TimeSeriesChart)
+          return;
+
+        const canvas = this.$refs?.chartCanvas;
+        if (!canvas) return;
+
+        this.destroyChart();
+
+        if (this.measurements.length === 0) return;
+
+        this.chartInstance = window.TimeSeriesChart.create(
+          canvas,
+          this.chartOptions,
+        );
+      },
+
+      updateChart() {
+        if (!this.chartInstance || !this.isViewMode) return;
+        window.TimeSeriesChart.update(
+          this.chartInstance,
+          this.measurements,
+          this.chartOptions,
+        );
+      },
+
+      destroyChart() {
+        if (this.chartInstance) {
+          window.TimeSeriesChart.destroy(this.chartInstance);
+          this.chartInstance = null;
+        }
+      },
+
+      // Alpine lifecycle hooks
+      init() {
+        if (this.isViewMode && window.Chart) {
+          // Use $nextTick to ensure DOM is ready
+          this.$nextTick(() => {
+            this.initChart();
+          });
+
+          // Watch measurements array for changes and update chart
+          this.$watch("data." + this.fieldName, () => {
+            if (this.chartInstance) {
+              this.updateChart();
+            }
+          });
+        }
+      },
+    }),
+  );
+});

static/js/alpine/pathway.js

@@ -21,6 +21,7 @@ document.addEventListener('alpine:init', () => {
   Alpine.data('pathwayViewer', (config) => ({
     status: config.status,
     modified: config.modified,
+    modifiedDate: null,
     statusUrl: config.statusUrl,
     emptyDueToThreshold: config.emptyDueToThreshold === "True",
     showUpdateNotice: false,
@@ -39,6 +40,8 @@ document.addEventListener('alpine:init', () => {
     },
 
     init() {
+      this.modifiedDate = this.parseDate(this.modified);
+
       if (this.status === 'running') {
         this.startPolling();
       }
@@ -66,26 +69,39 @@ document.addEventListener('alpine:init', () => {
           this.showEmptyDueToThresholdNotice = true;
         }
 
-        if (data.modified > this.modified) {
-          if (!this.emptyDueToThreshold) {
-            this.showUpdateNotice = true;
-            this.updateMessage = this.getUpdateMessage(data.status);
-          }
+        const nextModifiedDate = this.parseDate(data.modified);
+        const modifiedChanged = this.hasNewerTimestamp(nextModifiedDate, this.modifiedDate);
+        const statusChanged = data.status !== this.status;
+
+        if ((modifiedChanged || statusChanged) && !this.emptyDueToThreshold) {
+          this.showUpdateNotice = true;
+          this.updateMessage = this.getUpdateMessage(data.status, modifiedChanged, statusChanged);
         }
 
-        if (data.status !== 'running') {
-          this.status = data.status;
-          if (this.pollInterval) {
-            clearInterval(this.pollInterval);
-            this.pollInterval = null;
-          }
+        this.modified = data.modified;
+        this.modifiedDate = nextModifiedDate;
+        this.status = data.status;
+
+        if (data.status !== 'running' && this.pollInterval) {
+          clearInterval(this.pollInterval);
+          this.pollInterval = null;
         }
       } catch (err) {
         console.error('Polling error:', err);
       }
     },
 
-    getUpdateMessage(status) {
+    getUpdateMessage(status, modifiedChanged, statusChanged) {
+      // Prefer explicit status change messaging, otherwise fall back to modified change copy
+      if (statusChanged) {
+        if (status === 'completed') {
+          return 'Prediction completed. Reload the page to see the updated Pathway.';
+        }
+        if (status === 'failed') {
+          return 'Prediction failed. Reload the page to see the latest status.';
+        }
+      }
+
       let msg = 'Prediction ';
 
       if (status === 'running') {
@@ -99,6 +115,18 @@ document.addEventListener('alpine:init', () => {
       return msg;
     },
 
+    parseDate(dateString) {
+      // Normalize "YYYY-MM-DD HH:mm:ss" into an ISO-compatible string to avoid locale issues
+      if (!dateString) return null;
+      return new Date(dateString.replace(' ', 'T'));
+    },
+
+    hasNewerTimestamp(nextDate, currentDate) {
+      if (!nextDate) return false;
+      if (!currentDate) return true;
+      return nextDate.getTime() > currentDate.getTime();
+    },
+
     reloadPage() {
       location.reload();
     }

static/js/api/additional-information.js

@@ -0,0 +1,419 @@
+/**
+ * Unified API client for Additional Information endpoints
+ * Provides consistent error handling, logging, and CRUD operations
+ */
+window.AdditionalInformationApi = {
+  // Configuration
+  _debug: false,
+
+  /**
+   * Enable or disable debug logging
+   * @param {boolean} enabled - Whether to enable debug mode
+   */
+  setDebug(enabled) {
+    this._debug = enabled;
+  },
+
+  /**
+   * Internal logging helper
+   * @private
+   */
+  _log(action, data) {
+    if (this._debug) {
+      console.log(`[AdditionalInformationApi] ${action}:`, data);
+    }
+  },
+  //FIXME: this has the side effect of users not being able to explicitly set an empty string for a field.
+  /**
+   * Remove empty strings from payload recursively
+   * @param {any} value
+   * @returns {any}
+   */
+  sanitizePayload(value) {
+    if (Array.isArray(value)) {
+      return value
+        .map((item) => this.sanitizePayload(item))
+        .filter((item) => item !== "");
+    }
+
+    if (value && typeof value === "object") {
+      const cleaned = {};
+      for (const [key, item] of Object.entries(value)) {
+        if (item === "") continue;
+        cleaned[key] = this.sanitizePayload(item);
+      }
+      return cleaned;
+    }
+
+    return value;
+  },
+
+  /**
+   * Get CSRF token from meta tag
+   * @returns {string} CSRF token
+   */
+  getCsrfToken() {
+    return document.querySelector("[name=csrf-token]")?.content || "";
+  },
+
+  /**
+   * Build headers for API requests
+   * @private
+   */
+  _buildHeaders(includeContentType = true) {
+    const headers = {
+      "X-CSRFToken": this.getCsrfToken(),
+    };
+    if (includeContentType) {
+      headers["Content-Type"] = "application/json";
+    }
+    return headers;
+  },
+
+  /**
+   * Handle API response with consistent error handling
+   * @private
+   */
+  async _handleResponse(response, action) {
+    if (!response.ok) {
+      let errorData;
+      try {
+        errorData = await response.json();
+      } catch {
+        errorData = { error: response.statusText };
+      }
+
+      // Try to parse the error if it's a JSON string
+      let parsedError = errorData;
+      const errorStr = errorData.detail || errorData.error;
+      if (typeof errorStr === "string" && errorStr.startsWith("{")) {
+        try {
+          parsedError = JSON.parse(errorStr);
+        } catch {
+          // Not JSON, use as-is
+        }
+      }
+
+      // If it's a structured validation error, throw with field errors
+      if (parsedError.type === "validation_error" && parsedError.field_errors) {
+        this._log(`${action} VALIDATION ERROR`, parsedError);
+        const error = new Error(parsedError.message || "Validation failed");
+        error.fieldErrors = parsedError.field_errors;
+        error.isValidationError = true;
+        throw error;
+      }
+
+      // General error
+      const errorMsg =
+        parsedError.message ||
+        parsedError.error ||
+        parsedError.detail ||
+        `${action} failed: ${response.statusText}`;
+      this._log(`${action} ERROR`, {
+        status: response.status,
+        error: errorMsg,
+      });
+      throw new Error(errorMsg);
+    }
+
+    const data = await response.json();
+    this._log(`${action} SUCCESS`, data);
+    return data;
+  },
+
+  /**
+   * Load all available schemas
+   * @returns {Promise<Object>} Object with schema definitions
+   */
+  async loadSchemas() {
+    this._log("loadSchemas", "Starting...");
+    const response = await fetch("/api/v1/information/schema/");
+    return this._handleResponse(response, "loadSchemas");
+  },
+
+  /**
+   * Load additional information items for a scenario
+   * @param {string} scenarioUuid - UUID of the scenario
+   * @returns {Promise<Array>} Array of additional information items
+   */
+  async loadItems(scenarioUuid) {
+    this._log("loadItems", { scenarioUuid });
+    const response = await fetch(
+      `/api/v1/scenario/${scenarioUuid}/information/`,
+    );
+    return this._handleResponse(response, "loadItems");
+  },
+
+  /**
+   * Load both schemas and items in parallel
+   * @param {string} scenarioUuid - UUID of the scenario
+   * @returns {Promise<{schemas: Object, items: Array}>}
+   */
+  async loadSchemasAndItems(scenarioUuid) {
+    this._log("loadSchemasAndItems", { scenarioUuid });
+
+    const [schemas, items] = await Promise.all([
+      this.loadSchemas(),
+      this.loadItems(scenarioUuid),
+    ]);
+
+    return { schemas, items };
+  },
+
+  /**
+   * Create new additional information for a scenario
+   * @param {string} scenarioUuid - UUID of the scenario
+   * @param {string} modelName - Name/type of the additional information model
+   * @param {Object} data - Data for the new item
+   * @returns {Promise<{status: string, uuid: string}>}
+   */
+  async createItem(scenarioUuid, modelName, data) {
+    const sanitizedData = this.sanitizePayload(data);
+    this._log("createItem", { scenarioUuid, modelName, data: sanitizedData });
+
+    // Normalize model name to lowercase
+    const normalizedName = modelName.toLowerCase();
+
+    const response = await fetch(
+      `/api/v1/scenario/${scenarioUuid}/information/${normalizedName}/`,
+      {
+        method: "POST",
+        headers: this._buildHeaders(),
+        body: JSON.stringify(sanitizedData),
+      },
+    );
+
+    return this._handleResponse(response, "createItem");
+  },
+
+  /**
+   * Delete additional information from a scenario
+   * @param {string} scenarioUuid - UUID of the scenario
+   * @param {string} itemUuid - UUID of the item to delete
+   * @returns {Promise<{status: string}>}
+   */
+  async deleteItem(scenarioUuid, itemUuid) {
+    this._log("deleteItem", { scenarioUuid, itemUuid });
+
+    const response = await fetch(
+      `/api/v1/scenario/${scenarioUuid}/information/item/${itemUuid}/`,
+      {
+        method: "DELETE",
+        headers: this._buildHeaders(false),
+      },
+    );
+
+    return this._handleResponse(response, "deleteItem");
+  },
+
+  /**
+   * Update existing additional information
+   * Tries PATCH first, falls back to delete+recreate if not supported
+   * @param {string} scenarioUuid - UUID of the scenario
+   * @param {Object} item - Item object with uuid, type, and data properties
+   * @returns {Promise<{status: string, uuid: string}>}
+   */
+  async updateItem(scenarioUuid, item) {
+    const sanitizedData = this.sanitizePayload(item.data);
+    this._log("updateItem", {
+      scenarioUuid,
+      item: { ...item, data: sanitizedData },
+    });
+
+    const { uuid, type } = item;
+
+    // Try PATCH first (preferred method - preserves UUID)
+    const response = await fetch(
+      `/api/v1/scenario/${scenarioUuid}/information/item/${uuid}/`,
+      {
+        method: "PATCH",
+        headers: this._buildHeaders(),
+        body: JSON.stringify(sanitizedData),
+      },
+    );
+
+    if (response.status === 405) {
+      // PATCH not supported, fall back to delete+recreate
+      this._log(
+        "updateItem",
+        "PATCH not supported, falling back to delete+recreate",
+      );
+      await this.deleteItem(scenarioUuid, uuid);
+      return await this.createItem(scenarioUuid, type, sanitizedData);
+    }
+
+    return this._handleResponse(response, "updateItem");
+  },
+
+  /**
+   * Update multiple items sequentially to avoid race conditions
+   * @param {string} scenarioUuid - UUID of the scenario
+   * @param {Array<Object>} items - Array of items to update
+   * @returns {Promise<Array>} Array of results with success status
+   */
+  async updateItems(scenarioUuid, items) {
+    this._log("updateItems", { scenarioUuid, itemCount: items.length });
+
+    const results = [];
+    for (const item of items) {
+      try {
+        const result = await this.updateItem(scenarioUuid, item);
+        results.push({
+          success: true,
+          oldUuid: item.uuid,
+          newUuid: result.uuid,
+        });
+      } catch (error) {
+        results.push({
+          success: false,
+          oldUuid: item.uuid,
+          error: error.message,
+          fieldErrors: error.fieldErrors,
+          isValidationError: error.isValidationError,
+        });
+      }
+    }
+
+    const failed = results.filter((r) => !r.success);
+    if (failed.length > 0) {
+      // If all failures are validation errors, return all validation errors for display
+      const validationErrors = failed.filter((f) => f.isValidationError);
+      if (validationErrors.length === failed.length) {
+        // All failures are validation errors - return all field errors by item UUID
+        const allFieldErrors = {};
+        validationErrors.forEach((ve) => {
+          allFieldErrors[ve.oldUuid] = ve.fieldErrors || {};
+        });
+        const error = new Error(
+          `${failed.length} item(s) have validation errors. Please correct them.`,
+        );
+        error.fieldErrors = allFieldErrors; // Map of UUID -> field errors
+        error.isValidationError = true;
+        error.isMultipleErrors = true; // Flag indicating multiple items have errors
+        throw error;
+      }
+      // Multiple failures or mixed errors - show count
+      throw new Error(
+        `Failed to update ${failed.length} item(s). Please check the form for errors.`,
+      );
+    }
+
+    return results;
+  },
+
+  /**
+   * Create a new scenario with optional additional information
+   * @param {string} packageUuid - UUID of the package
+   * @param {Object} payload - Scenario data matching ScenarioCreateSchema
+   * @param {string} payload.name - Scenario name (required)
+   * @param {string} payload.description - Scenario description (optional, default: "")
+   * @param {string} payload.scenario_date - Scenario date (optional, default: "No date")
+   * @param {string} payload.scenario_type - Scenario type (optional, default: "Not specified")
+   * @param {Array} payload.additional_information - Array of additional information (optional, default: [])
+   * @returns {Promise<{uuid, url, name, description, review_status, package}>}
+   */
+  async createScenario(packageUuid, payload) {
+    this._log("createScenario", { packageUuid, payload });
+    const response = await fetch(`/api/v1/package/${packageUuid}/scenario/`, {
+      method: "POST",
+      headers: this._buildHeaders(),
+      body: JSON.stringify(payload),
+    });
+    return this._handleResponse(response, "createScenario");
+  },
+
+  /**
+   * Load all available group names
+   * @returns {Promise<{groups: string[]}>}
+   */
+  async loadGroups() {
+    this._log("loadGroups", "Starting...");
+    const response = await fetch("/api/v1/information/groups/");
+    return this._handleResponse(response, "loadGroups");
+  },
+
+  /**
+   * Load model definitions for a specific group
+   * @param {string} groupName - One of 'soil', 'sludge', 'sediment'
+   * @returns {Promise<Object>} Object with subcategories as keys and arrays of model info
+   */
+  async loadGroupModels(groupName) {
+    this._log("loadGroupModels", { groupName });
+    const response = await fetch(`/api/v1/information/groups/${groupName}/`);
+    return this._handleResponse(response, `loadGroupModels-${groupName}`);
+  },
+
+  /**
+   * Load model information for multiple groups in parallel
+   * @param {Array<string>} groupNames - Defaults to ['soil', 'sludge', 'sediment']
+   * @returns {Promise<Object>} Object with group names as keys
+   */
+  async loadGroupsWithModels(groupNames = ["soil", "sludge", "sediment"]) {
+    this._log("loadGroupsWithModels", { groupNames });
+
+    const results = {};
+    const promises = groupNames.map(async (groupName) => {
+      try {
+        results[groupName] = await this.loadGroupModels(groupName);
+      } catch (err) {
+        this._log(`loadGroupsWithModels-${groupName} ERROR`, err);
+        results[groupName] = {};
+      }
+    });
+
+    await Promise.all(promises);
+    return results;
+  },
+
+  /**
+   * Helper to organize schemas by group based on group model information
+   * @param {Object} schemas - Full schema map from loadSchemas()
+   * @param {Object} groupModelsData - Group models data from loadGroupsWithModels()
+   * @returns {Object} Object with group names as keys and filtered schemas as values
+   */
+  organizeSchemasByGroup(schemas, groupModelsData) {
+    this._log("organizeSchemasByGroup", {
+      schemaCount: Object.keys(schemas).length,
+      groupCount: Object.keys(groupModelsData).length,
+    });
+
+    const organized = {};
+
+    for (const groupName in groupModelsData) {
+      organized[groupName] = {};
+      const groupData = groupModelsData[groupName];
+
+      // Iterate through subcategories in the group
+      for (const subcategory in groupData) {
+        for (const model of groupData[subcategory]) {
+          // Look up schema by lowercase model name
+          if (schemas[model.name]) {
+            organized[groupName][model.name] = schemas[model.name];
+          }
+        }
+      }
+    }
+
+    return organized;
+  },
+
+  /**
+   * Convenience method that loads schemas and organizes them by group in one call
+   * @param {Array<string>} groupNames - Defaults to ['soil', 'sludge', 'sediment']
+   * @returns {Promise<{schemas, groupSchemas, groupModels}>}
+   */
+  async loadSchemasWithGroups(groupNames = ["soil", "sludge", "sediment"]) {
+    this._log("loadSchemasWithGroups", { groupNames });
+
+    // Load schemas and all groups in parallel
+    const [schemas, groupModels] = await Promise.all([
+      this.loadSchemas(),
+      this.loadGroupsWithModels(groupNames),
+    ]);
+
+    // Organize schemas by group
+    const groupSchemas = this.organizeSchemasByGroup(schemas, groupModels);
+
+    return { schemas, groupSchemas, groupModels };
+  },
+};

static/js/pw.js

@@ -361,6 +361,83 @@ function draw(pathway, elem) {
 
     function node_popup(n) {
         popupContent = "";
+
+        if (timeseriesViewEnabled && n.timeseries && n.timeseries.measurements) {
+            for (var s of n.scenarios) {
+                popupContent += "<a href='" + s.url + "'>" + s.name + "</a><br>";
+            }
+
+            popupContent += '<div style="width:100%;height:120px"><canvas id="ts-popover-canvas"></canvas></div>';
+            const tsMeasurements = n.timeseries.measurements;
+            setTimeout(() => {
+                const canvas = document.getElementById('ts-popover-canvas');
+                if (canvas && window.Chart) {
+                    const valid = tsMeasurements
+                        .filter(m => m.timestamp != null && m.value != null)
+                        .map(m => ({ ...m, timestamp: typeof m.timestamp === 'number' ? m.timestamp : new Date(m.timestamp).getTime() }))
+                        .sort((a, b) => a.timestamp - b.timestamp);
+
+                    const datasets = [];
+
+                    // Error band (lower + upper with fill between)
+                    const withErrors = valid.filter(m => m.error != null && m.error > 0);
+                    if (withErrors.length > 0) {
+                        datasets.push({
+                            data: withErrors.map(m => ({ x: m.timestamp, y: m.value - m.error })),
+                            borderColor: 'rgba(59,130,246,0.3)',
+                            backgroundColor: 'rgba(59,130,246,0.15)',
+                            pointRadius: 0,
+                            fill: false,
+                            tension: 0.1,
+                        });
+                        datasets.push({
+                            data: withErrors.map(m => ({ x: m.timestamp, y: m.value + m.error })),
+                            borderColor: 'rgba(59,130,246,0.3)',
+                            backgroundColor: 'rgba(59,130,246,0.15)',
+                            pointRadius: 0,
+                            fill: '-1',
+                            tension: 0.1,
+                        });
+                    }
+
+                    // Main value line
+                    datasets.push({
+                        data: valid.map(m => ({ x: m.timestamp, y: m.value })),
+                        borderColor: 'rgb(59,130,246)',
+                        pointRadius: 0,
+                        tension: 0.1,
+                        fill: false,
+                    });
+
+                    new Chart(canvas.getContext('2d'), {
+                        type: 'line',
+                        data: { datasets },
+                        options: {
+                            responsive: true,
+                            maintainAspectRatio: false,
+                            plugins: {
+                                legend: { display: false },
+                                tooltip: { enabled: false },
+                            },
+                            scales: {
+                                x: {
+                                    type: 'linear',
+                                    ticks: { font: { size: 10 } },
+                                    title: { display: false },
+                                },
+                                y: {
+                                    ticks: { font: { size: 10 } },
+                                    title: { display: false },
+                                },
+                            },
+                        },
+                    });
+                }
+            }, 0);
+
+            return popupContent;
+        }
+
         if (n.stereo_removed) {
             popupContent += "<span class='alert alert-warning alert-soft'>Removed stereochemistry for prediction</span>";
         }
@@ -376,6 +453,29 @@ function draw(pathway, elem) {
             }
         }
 
+        if (predictedPropertyViewEnabled) {
+
+            var tempContent = "";
+
+            if (Object.keys(n.predicted_properties).length > 0) {
+
+                if ("PepperPrediction" in n.predicted_properties) {
+                    // TODO needs to be generic once we store it as AddInf
+                    for (var s of n.predicted_properties["PepperPrediction"]) {
+                        if (s["mean"] != null) {
+                            tempContent += "<b>DT50 predicted via Pepper:</b> " + s["mean"].toFixed(2) + "<br>"
+                        }
+                    }
+                }
+            }
+
+            if (tempContent === "") {
+                tempContent = "<b>No predicted properties for this Node</b><br>";
+            }
+
+            popupContent += tempContent
+        }
+
         popupContent += "<img src='" + n.image + "'><br>"
         if (n.scenarios.length > 0) {
             popupContent += '<b>Half-lives and related scenarios:</b><br>'
@@ -393,7 +493,13 @@ function draw(pathway, elem) {
     }
 
     function edge_popup(e) {
-        popupContent = "<a href='" + e.url + "'>" + e.name + "</a><br>";
+        popupContent = "<a href='" + e.url + "'>" + e.name + "</a><br><br>";
+
+        if (e.reaction.rules) {
+            for (var rule of e.reaction.rules) {
+                popupContent += "Rule <a href='" + rule.url + "'>" + rule.name + "</a><br>";
+            }
+        }
 
         if (e.app_domain) {
             adcontent = "<p>";
@@ -498,7 +604,8 @@ function draw(pathway, elem) {
         .enter().append("line")
         // Check if target is pseudo and draw marker only if not pseudo
         .attr("class", d => d.target.pseudo ? "link_no_arrow" : "link")
-        .attr("marker-end", d => d.target.pseudo ? '' : 'url(#arrow)')
+        .attr("marker-end", d => d.target.pseudo ? '' : d.multi_step ? 'url(#doublearrow)' : 'url(#arrow)')
+
 
     // add element to links array
     link.each(function (d) {

static/js/utils/timeseries-chart.js

@@ -0,0 +1,351 @@
+/**
+ * TimeSeriesChart Utility
+ *
+ * Provides chart rendering capabilities for time series data with error bounds.
+ * Uses Chart.js to create interactive and static visualizations.
+ *
+ * Usage:
+ *   const chart = window.TimeSeriesChart.create(canvas, {
+ *     measurements: [...],
+ *     xAxisLabel: "Time",
+ *     yAxisLabel: "Concentration",
+ *     xAxisUnit: "days",
+ *     yAxisUnit: "mg/L"
+ *   });
+ *
+ *   window.TimeSeriesChart.update(chart, newMeasurements, options);
+ *   window.TimeSeriesChart.destroy(chart);
+ */
+window.TimeSeriesChart = {
+  // === PUBLIC API ===
+
+  /**
+   * Create an interactive time series chart
+   *
+   * @param {HTMLCanvasElement} canvas - Canvas element to render chart on
+   * @param {Object} options - Chart configuration options
+   * @param {Array} options.measurements - Array of measurement objects with timestamp, value, error, note
+   * @param {string} options.xAxisLabel - Label for x-axis (default: "Time")
+   * @param {string} options.yAxisLabel - Label for y-axis (default: "Value")
+   * @param {string} options.xAxisUnit - Unit for x-axis (default: "")
+   * @param {string} options.yAxisUnit - Unit for y-axis (default: "")
+   * @returns {Chart|null} Chart.js instance or null if creation failed
+   */
+  create(canvas, options = {}) {
+    if (!this._validateCanvas(canvas)) return null;
+    if (!window.Chart) {
+      console.warn("Chart.js is not loaded");
+      return null;
+    }
+
+    const ctx = canvas.getContext("2d");
+    if (!ctx) return null;
+
+    const chartData = this._transformData(options.measurements || [], options);
+
+    if (chartData.datasets.length === 0) {
+      return null; // No data to display
+    }
+
+    const config = this._buildConfig(chartData, options);
+
+    return new Chart(ctx, config);
+  },
+
+  /**
+   * Update an existing chart with new data
+   *
+   * @param {Chart} chartInstance - Chart.js instance to update
+   * @param {Array} measurements - New measurements array
+   * @param {Object} options - Chart configuration options
+   */
+  update(chartInstance, measurements, options = {}) {
+    if (!chartInstance) return;
+
+    const chartData = this._transformData(measurements || [], options);
+
+    chartInstance.data.datasets = chartData.datasets;
+    chartInstance.options.scales.x.title.text = chartData.xAxisLabel;
+    chartInstance.options.scales.y.title.text = chartData.yAxisLabel;
+    chartInstance.update("none");
+  },
+
+  /**
+   * Destroy chart instance and cleanup
+   *
+   * @param {Chart} chartInstance - Chart.js instance to destroy
+   */
+  destroy(chartInstance) {
+    if (chartInstance && typeof chartInstance.destroy === "function") {
+      chartInstance.destroy();
+    }
+  },
+
+  // === PRIVATE HELPERS ===
+
+  /**
+   * Transform measurements into Chart.js datasets
+   * @private
+   */
+  _transformData(measurements, options) {
+    const preparedData = this._prepareData(measurements);
+
+    if (preparedData.length === 0) {
+      return { datasets: [], xAxisLabel: "Time", yAxisLabel: "Value" };
+    }
+
+    const xAxisLabel = options.xAxisLabel || "Time";
+    const yAxisLabel = options.yAxisLabel || "Value";
+    const xAxisUnit = options.xAxisUnit || "";
+    const yAxisUnit = options.yAxisUnit || "";
+
+    const datasets = [];
+
+    // Error bounds datasets FIRST (if errors exist) - renders as background
+    const errorDatasets = this._buildErrorDatasets(preparedData);
+    if (errorDatasets.length > 0) {
+      datasets.push(...errorDatasets);
+    }
+
+    // Main line dataset LAST - renders on top
+    datasets.push(this._buildMainDataset(preparedData, yAxisLabel));
+
+    return {
+      datasets: datasets,
+      xAxisLabel: this._formatAxisLabel(xAxisLabel, xAxisUnit),
+      yAxisLabel: this._formatAxisLabel(yAxisLabel, yAxisUnit),
+    };
+  },
+
+  /**
+   * Prepare and validate measurements data
+   * @private
+   */
+  _prepareData(measurements) {
+    return measurements
+      .filter(
+        (m) => m.timestamp != null && m.value != null,
+      )
+      .map((m) => {
+        // Normalize timestamp - handle both numeric and date strings
+        let timestamp;
+        if (typeof m.timestamp === "number") {
+          timestamp = m.timestamp;
+        } else {
+          timestamp = new Date(m.timestamp).getTime();
+        }
+        return {
+          ...m,
+          timestamp: timestamp,
+        };
+      })
+      .sort((a, b) => a.timestamp - b.timestamp);
+  },
+
+  /**
+   * Build main line dataset
+   * @private
+   */
+  _buildMainDataset(validMeasurements, yAxisLabel) {
+    return {
+      label: yAxisLabel,
+      data: validMeasurements.map((m) => ({
+        x: m.timestamp,
+        y: m.value,
+        error: m.error || null,
+      })),
+      borderColor: "rgb(59, 130, 246)",
+      backgroundColor: "rgba(59, 130, 246, 0.1)",
+      tension: 0.1,
+      pointRadius: 0, // Hide individual points
+      pointHoverRadius: 6,
+      fill: false,
+    };
+  },
+
+  /**
+   * Build error bound datasets (upper and lower)
+   * @private
+   */
+  _buildErrorDatasets(validMeasurements) {
+    const hasErrors = validMeasurements.some(
+      (m) => m.error !== null && m.error !== undefined && m.error > 0,
+    );
+
+    if (!hasErrors) return [];
+
+    const measurementsWithErrors = validMeasurements.filter(
+      (m) => m.error !== null && m.error !== undefined && m.error > 0,
+    );
+
+    return [
+      // Lower error bound - FIRST (bottom layer)
+      {
+        label: "Error (lower)",
+        data: measurementsWithErrors.map((m) => ({
+          x: m.timestamp,
+          y: m.value - m.error,
+        })),
+        borderColor: "rgba(59, 130, 246, 0.3)",
+        backgroundColor: "rgba(59, 130, 246, 0.15)",
+        pointRadius: 0,
+        fill: false,
+        tension: 0.1,
+      },
+      // Upper error bound - SECOND (fill back to lower)
+      {
+        label: "Error (upper)",
+        data: measurementsWithErrors.map((m) => ({
+          x: m.timestamp,
+          y: m.value + m.error,
+        })),
+        borderColor: "rgba(59, 130, 246, 0.3)",
+        backgroundColor: "rgba(59, 130, 246, 0.15)",
+        pointRadius: 0,
+        fill: "-1", // Fill back to previous dataset (lower bound)
+        tension: 0.1,
+      },
+    ];
+  },
+
+  /**
+   * Build complete Chart.js configuration
+   * @private
+   */
+  _buildConfig(chartData, options) {
+    return {
+      type: "line",
+      data: { datasets: chartData.datasets },
+      options: {
+        responsive: true,
+        maintainAspectRatio: false,
+        interaction: {
+          intersect: false,
+          mode: "index",
+        },
+        plugins: {
+          legend: {
+            display: false,
+          },
+          tooltip: this._buildTooltipConfig(
+            options.xAxisUnit || "",
+            options.yAxisUnit || "",
+          ),
+        },
+        scales: this._buildScalesConfig(
+          chartData.xAxisLabel,
+          chartData.yAxisLabel,
+        ),
+      },
+    };
+  },
+
+  /**
+   * Build tooltip configuration with custom callbacks
+   * @private
+   */
+  _buildTooltipConfig(xAxisUnit, yAxisUnit) {
+    return {
+      enabled: true,
+      callbacks: {
+        title: (contexts) => {
+          // Show timestamp
+          const context = contexts[0];
+          if (!context) return "Measurement";
+          const timestamp = context.parsed.x;
+          return xAxisUnit
+            ? `Time: ${timestamp} ${xAxisUnit}`
+            : `Time: ${timestamp}`;
+        },
+        label: (context) => {
+          // Show value with unit
+          try {
+            const value = context.parsed.y;
+            if (value === null || value === undefined) {
+              return `${context.dataset.label || "Value"}: N/A`;
+            }
+
+            const valueStr = yAxisUnit
+              ? `${value} ${yAxisUnit}`
+              : String(value);
+            return `${context.dataset.label || "Value"}: ${valueStr}`;
+          } catch (e) {
+            console.error("Tooltip label error:", e);
+            return `${context.dataset.label || "Value"}: ${context.parsed.y ?? "N/A"}`;
+          }
+        },
+        afterLabel: (context) => {
+          // Show error information
+          try {
+            const point = context.raw;
+            // Main line is now the last dataset (after error bounds if they exist)
+            const isMainDataset = context.dataset.label &&
+                                 !context.dataset.label.startsWith("Error");
+            if (!point || !isMainDataset) return null;
+
+            const lines = [];
+
+            // Show error if available
+            if (
+              point.error !== null &&
+              point.error !== undefined &&
+              point.error > 0
+            ) {
+              const errorStr = yAxisUnit
+                ? `±${point.error.toFixed(4)} ${yAxisUnit}`
+                : `±${point.error.toFixed(4)}`;
+              lines.push(`Error: ${errorStr}`);
+            }
+
+            return lines.length > 0 ? lines : null;
+          } catch (e) {
+            console.error("Tooltip afterLabel error:", e);
+            return null;
+          }
+        },
+      },
+    };
+  },
+
+  /**
+   * Build scales configuration
+   * @private
+   */
+  _buildScalesConfig(xAxisLabel, yAxisLabel) {
+    return {
+      x: {
+        type: "linear",
+        title: {
+          display: true,
+          text: xAxisLabel || "Time",
+        },
+      },
+      y: {
+        title: {
+          display: true,
+          text: yAxisLabel || "Value",
+        },
+      },
+    };
+  },
+
+  /**
+   * Format axis label with unit
+   * @private
+   */
+  _formatAxisLabel(label, unit) {
+    return unit ? `${label} (${unit})` : label;
+  },
+
+  /**
+   * Validate canvas element
+   * @private
+   */
+  _validateCanvas(canvas) {
+    if (!canvas || !(canvas instanceof HTMLCanvasElement)) {
+      console.warn("Invalid canvas element provided to TimeSeriesChart");
+      return false;
+    }
+    return true;
+  },
+};

templates/actions/objects/scenario.html

@@ -1,4 +1,12 @@
 {% if meta.can_edit %}
+  <li>
+    <a
+      class="button"
+      onclick="document.getElementById('edit_scenario_modal').showModal(); return false;"
+    >
+      <i class="glyphicon glyphicon-trash"></i> Edit Scenario</a
+    >
+  </li>
   <li>
     <a
       class="button"

templates/batch_predict_pathway.html

@@ -9,6 +9,39 @@
       <input type="hidden" name="job-name" value="batch-predict" />
 
       <fieldset class="flex flex-col gap-4 md:flex-3/4">
+        <!-- CSV Upload Section -->
+        <div class="mb-6 rounded-lg border-2 border-dashed border-base-300 p-6">
+          <div class="flex flex-col gap-4">
+            <div
+              class="flex flex-col gap-3 sm:flex-row sm:items-center sm:justify-between"
+            >
+              <div class="flex-1">
+                <h3 class="text-base font-medium text-base-content mb-1">
+                  Load from CSV
+                </h3>
+                <p class="text-sm text-base-content/70">
+                  Upload a CSV file with SMILES and name columns, or insert
+                  manually in the table below
+                </p>
+              </div>
+              <div class="flex-shrink-0">
+                <input
+                  type="file"
+                  id="csv-file"
+                  accept=".csv,.txt"
+                  class="file-input file-input-bordered file-input-sm w-full sm:w-auto"
+                />
+              </div>
+            </div>
+            <div
+              class="text-xs text-base-content/50 border-t border-base-300 pt-3"
+            >
+              <strong>Format:</strong> First column = SMILES, Second column =
+              Name (headers optional) • Maximum 30 rows
+            </div>
+          </div>
+        </div>
+
         <table class="table table-zebra w-full">
           <thead>
             <tr>
@@ -113,10 +146,16 @@
   <script>
     const tableBody = document.getElementById("smiles-table-body");
     const addRowBtn = document.getElementById("add-row-btn");
+    const csvFileInput = document.getElementById("csv-file");
     const form = document.getElementById("smiles-form");
     const hiddenField = document.getElementById("substrates");
 
-    addRowBtn.addEventListener("click", () => {
+    // Function to create a new table row
+    function createTableRow(
+      smilesValue = "",
+      nameValue = "",
+      placeholder = true,
+    ) {
       const row = document.createElement("tr");
 
       const tdSmiles = document.createElement("td");
@@ -125,19 +164,147 @@
       const smilesInput = document.createElement("input");
       smilesInput.type = "text";
       smilesInput.className = "input input-bordered w-full smiles-input";
-      smilesInput.placeholder = "SMILES";
+      smilesInput.placeholder = placeholder ? "SMILES" : "";
+      smilesInput.value = smilesValue;
 
       const nameInput = document.createElement("input");
       nameInput.type = "text";
       nameInput.className = "input input-bordered w-full name-input";
-      nameInput.placeholder = "Name";
+      nameInput.placeholder = placeholder ? "Name" : "";
+      nameInput.value = nameValue;
 
-      tdSmiles.appendChild(smilesInput);
-      tdName.appendChild(nameInput);
+      const smilesLabel = document.createElement("label");
+      smilesLabel.appendChild(smilesInput);
+      tdSmiles.appendChild(smilesLabel);
+
+      const nameLabel = document.createElement("label");
+      nameLabel.appendChild(nameInput);
+      tdName.appendChild(nameLabel);
 
       row.appendChild(tdSmiles);
       row.appendChild(tdName);
 
+      return row;
+    }
+
+    // Function to clear the table
+    function clearTable() {
+      tableBody.innerHTML = "";
+    }
+
+    // Function to populate table from CSV data
+    function populateTableFromCSV(csvData) {
+      const lines = csvData.trim().split("\n");
+      const maxRows = 30;
+
+      // Clear existing table
+      clearTable();
+
+      // Skip header row if it looks like headers
+      const startIndex =
+        lines.length > 0 &&
+        (lines[0].toLowerCase().includes("smiles") ||
+          lines[0].toLowerCase().includes("name"))
+          ? 1
+          : 0;
+
+      let rowCount = 0;
+      for (let i = startIndex; i < lines.length && rowCount < maxRows; i++) {
+        const line = lines[i].trim();
+        if (!line) continue;
+
+        // Parse CSV line - split by comma, first part is SMILES, rest is name
+        const firstCommaIndex = line.indexOf(",");
+        let smiles = "";
+        let name = "";
+
+        if (firstCommaIndex === -1) {
+          // No comma found, treat entire line as SMILES
+          smiles = line.trim().replace(/^"(.*)"$/, "$1");
+        } else {
+          // Split at first comma only
+          smiles = line
+            .substring(0, firstCommaIndex)
+            .trim()
+            .replace(/^"(.*)"$/, "$1");
+          name = line
+            .substring(firstCommaIndex + 1)
+            .trim()
+            .replace(/^"(.*)"$/, "$1");
+        }
+
+        // Skip empty rows
+        if (!smiles && !name) continue;
+
+        const row = createTableRow(smiles, name, false);
+        tableBody.appendChild(row);
+        rowCount++;
+      }
+
+      // Add at least one empty row if no data was loaded
+      if (rowCount === 0) {
+        const row = createTableRow();
+        tableBody.appendChild(row);
+      }
+
+      // Show success message
+      if (rowCount > 0) {
+        const message =
+          rowCount >= maxRows
+            ? `Loaded ${rowCount} rows (maximum reached)`
+            : `Loaded ${rowCount} rows from CSV`;
+
+        // Create temporary success notification
+        const notification = document.createElement("div");
+        notification.className = "alert alert-success mb-4";
+        notification.innerHTML = `
+      <svg xmlns="http://www.w3.org/2000/svg" class="stroke-current shrink-0 h-6 w-6" fill="none" viewBox="0 0 24 24">
+        <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z" />
+      </svg>
+      <span>${message}</span>
+    `;
+
+        // Insert notification before the table
+        const tableContainer = document.querySelector("table").parentNode;
+        tableContainer.insertBefore(
+          notification,
+          document.querySelector("table"),
+        );
+
+        // Remove notification after 3 seconds
+        setTimeout(() => {
+          if (notification.parentNode) {
+            notification.parentNode.removeChild(notification);
+          }
+        }, 3000);
+      }
+    }
+    // Handle CSV file selection
+    csvFileInput.addEventListener("change", function (event) {
+      const file = event.target.files[0];
+      if (!file) return;
+
+      // Check file type
+      if (!file.name.match(/\.(csv|txt)$/i)) {
+        alert("Please select a CSV or TXT file.");
+        return;
+      }
+
+      const reader = new FileReader();
+      reader.onload = function (e) {
+        try {
+          populateTableFromCSV(e.target.result);
+        } catch (error) {
+          console.error("Error parsing CSV:", error);
+          alert("Error parsing CSV file. Please check the file format.");
+        }
+      };
+      reader.readAsText(file);
+    });
+
+    // Handle add row button
+    addRowBtn.addEventListener("click", () => {
+      const row = createTableRow();
       tableBody.appendChild(row);
     });
 
@@ -154,7 +321,7 @@
         const smiles = smilesInputs[i].value.trim();
         const name = nameInputs[i]?.value.trim() ?? "";
 
-        // Skip emtpy rows
+        // Skip empty rows
         if (!smiles && !name) {
           continue;
         }

templates/collections/paginated_base.html

@@ -98,7 +98,7 @@
           class="mt-6"
           x-show="activeTab === 'reviewed' && !isEmpty"
           x-data="remotePaginatedList({
-              endpoint: '{{ api_endpoint }}?review_status=true',
+              endpoint: '{{ api_endpoint }}?review_status=true{% if entity_type == 'scenario' %}&exclude_related=true{% endif %}',
               instanceId: '{{ entity_type }}_reviewed',
               isReviewed: true,
               perPage: {{ per_page|default:50 }}
@@ -113,7 +113,7 @@
           class="mt-6"
           x-show="activeTab === 'unreviewed' && !isEmpty"
           x-data="remotePaginatedList({
-              endpoint: '{{ api_endpoint }}?review_status=false',
+              endpoint: '{{ api_endpoint }}?review_status=false{% if entity_type == 'scenario' %}&exclude_related=true{% endif %}',
               instanceId: '{{ entity_type }}_unreviewed',
               isReviewed: false,
               perPage: {{ per_page|default:50 }}