Commit Graph

96 Commits

Author SHA1 Message Date
c2d45917ce clean up d3_json 2025-10-31 09:38:41 +01:00
13ed86a780 [Feature] Identify Missing Rules (#177)
Fixes #97
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#177
2025-10-30 00:47:45 +13:00
f1b4c5aadb [Feature] Adding list_display to various django admin sites (#180)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#180
2025-10-29 22:26:28 +13:00
37e0e18a28 [Fix] Fixed Incremental Prediction Typo (#176)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#176
2025-10-28 23:29:08 +13:00
de44c22606 [Migration] Added missing Migration for JobLog (#175)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#175
2025-10-27 22:41:16 +13:00
a952c08469 [Feature] Basic logging of Jobs, Model Evaluation (#169)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#169
2025-10-27 22:34:05 +13:00
551cfc7768 [Enhancement] Create ML Models (#173)
## Changes

- Ability to change the threshold from a command line argument.
- Names of data packages included in model name
- Names of data, rule and eval packages included in the model description
- EnviFormer models are now viewable on the admin site
- Ignore CO2 for training and evaluating EnviFormer

Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#173
Reviewed-by: jebus <lorsbach@envipath.com>
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
2025-10-23 06:20:22 +13:00
8fda2577ee [Feature] Dump/Restore of enviFormer Models (#170)
Dump:
`./manage.py  dump_enviformer d544303c-a1ca-439d-b036-5e3413ce4a48 --output test.tar.gz`

Restore:
`./manage.py load_enviformer test.tar.gz 1062eb09-5ec7-4bdd-a8f2-ae0252eb4b06`

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#170
2025-10-22 10:39:22 +13:00
819a94aced [Fix] Catch Exception for Adding Structures / Show PubChem Substances (#168)
Fixes #163
Fixes #165

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#168
2025-10-22 01:13:06 +13:00
376fd65785 [Feature] ML model caching for reducing prediction overhead (#156)
The caching is now finished. The cache is created in `settings.py` giving us the most flexibility for using it in the future.

The cache is currently updated/accessed by `tasks.py/get_ml_model` which can be called from whatever task needs to access ml models in this way (currently, `predict` and `predict_simple`).

This implementation currently caches all ml models including the relative reasoning. If we don't want this and only want to cache enviFormer, i can change it to that. However, I don't think there is a harm in having the other models be cached as well.

Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#156
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
2025-10-16 08:58:36 +13:00
d5ebb23622 [Fix] AppDomain Leftovers (#161)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#161
beta_2025-10-15
2025-10-16 08:17:39 +13:00
93dd811e39 [Fix] Pathway SVG Export (#157)
Fixes #103

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#157
2025-10-16 02:25:30 +13:00
9a4735246f [Fix] Fix for sending mails (#160)
Captured by Sentry: https://envipath-limited.sentry.io/issues/66662009/?project=4509569727922256

```
SMTPSenderRefused
Level: Error
(504, b'5.5.2 <webmaster@localhost>: Sender address rejected: need fully-qualified address', 'webmaster@localhost')
```

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#160
2025-10-16 02:24:51 +13:00
1f863fdcd6 [Fix] Remove Scenarios from Objects (#159)
Fixes #155

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#159
2025-10-15 20:23:52 +13:00
1effaeb342 [Migration] EnzymeLink Migration (#158)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#158
2025-10-15 19:57:03 +13:00
386098b8a6 [Feature] EnzymeLink Annotations (#152)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#152
2025-10-15 19:35:26 +13:00
ef697ac5f5 [Fix] Added UZH Affiliation, Update UoA Images (#153)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#153
2025-10-15 19:25:23 +13:00
68a3f3b982 [Feature] Alias Support (#151)
Fixes #149

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#151
2025-10-09 23:14:34 +13:00
afeb56622c [Chore] Linted Files (#150)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#150
2025-10-09 07:25:13 +13:00
22f0bbe10b [Feature] Eval package evaluation
`evaluate_model` in `PackageBasedModel` and `EnviFormer` now use evaluation packages if any are present instead of the random splits.

Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#148
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
2025-10-08 19:03:21 +13:00
36879c266b [Feature] Documentation for development setup
## Summary

This PR improves the local development setup experience by adding Docker Compose and Makefile for streamlined setup.

## Changes

- **Added `docker-compose.yml`**: for one-command PostgreSQL database setup
- **Added `Makefile`**: Convenient shortcuts for common dev tasks (\`make setup\`, \`make dev\`, etc.)
- **Updated `README.md`**: Quick development setup instructions using Make
-
- **Added**: RDkit installation pain point documentation
- **Fixed**: Made Java feature properly dependent

## Why these changes?

The application uses PostgreSQL-specific features (\`ArrayField\`) and requires an anonymous user created by the bootstrap command. This PR makes the setup process trivial for new developers:

```bash
cp .env.local.example .env
make setup  # Starts DB, runs migrations, bootstraps data
make dev    # Starts development server
```

Java fix:
Moved global Java import to inline to avoid everyone having to configure the Java path.

Numerous changes to view and settings.
- Applied ruff-formatting

## Testing

Verified complete setup from scratch works with:
- PostgreSQL running in Docker
- All migrations applied
- Bootstrap data loaded successfully
- Anonymous user created
- The development server starts correctly.

Co-authored-by: Tobias O <tobias.olenyi@tum.de>
Co-authored-by: Tobias O <tobias.olenyi@envipath.com>
Co-authored-by: Liam <62733830+limmooo@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#143
Reviewed-by: jebus <lorsbach@envipath.com>
Reviewed-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-authored-by: t03i <mail+envipath@t03i.net>
Co-committed-by: t03i <mail+envipath@t03i.net>
2025-10-08 18:51:50 +13:00
c2c46fbfa7 [Migration] Added missing Migration for #141 (#147)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#147
2025-10-07 21:22:13 +13:00
d2f4fdc58a [Feature] Enviformer fine tuning and evaluation
## Changes
- I have finished the backend integration of EnviFormer (#19), this includes, dataset building, model finetuning, model evaluation and model prediction with the finetuned model.
- `PackageBasedModel` has been adjusted to be more abstract, this includes making the `_save_model` method and making `compute_averages` a static class function.
- I had to bump the python-version in `pyproject.toml` to >=3.12 from >=3.11 otherwise uv failed to install EnviFormer.
- The default EnviFormer loading during `settings.py` has been removed.

## Future Fix
I noticed you have a little bit of code in `PackageBasedModel` -> `evaluate_model` for using the `eval_packages` during evaluation instead of train/test splits on `data_packages`. It doesn't seem finished, I presume we want this for all models, so I will take care of that in a new branch/pullrequest after this request is merged.

Also, I haven't done anything for a POST request to finetune the model, I'm not sure if that is something we want now.

Co-authored-by: Liam Brydon <62733830+MyCreativityOutlet@users.noreply.github.com>
Reviewed-on: enviPath/enviPy#141
Reviewed-by: jebus <lorsbach@envipath.com>
Co-authored-by: liambrydon <lbry121@aucklanduni.ac.nz>
Co-committed-by: liambrydon <lbry121@aucklanduni.ac.nz>
2025-10-07 21:14:10 +13:00
3f2b046bd6 [Feature] More on Legacy API (#142)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#142
2025-10-03 00:07:30 +13:00
7ad4112343 [Feature] External Identifier/References
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#139
2025-10-02 00:40:00 +13:00
3f5bb76633 [Fix] Remove all Scenarios, catch empty SMILES, prevent default Package delete (#134)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#134
2025-09-30 19:10:57 +13:00
b757a07f91 [Misc] Performance improvements, SMIRKS Coverage, Minor Bugfixes (#132)
Bump Python Version to 3.12
Make use of "epauth" optional
Cache `srs` property of rules to speed up apply
Adjust view names for use of `reverse()`
Fix Views for Scenario Attachments
Added Simply Compare View/Template to identify differences between rdkit and ambit
Make migrations consistent with tests + compare
Fixes #76
Set default year for Scenario Modal
Fix html tags for package description
Added Tests for Pathway / Rule
Added remove stereo for apply

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#132
2025-09-26 19:33:03 +12:00
b5c759d74e [Feature] Register / Login / Logout View Testing (#126)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#126
beta_2025-09-18
2025-09-19 06:44:25 +12:00
b3079834c1 [Fix] Registering Issue (#125)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#125
2025-09-19 01:17:03 +12:00
f03ade0e94 [Migration] Multigen Eval Flag Migration (#123)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#123
2025-09-18 18:44:02 +12:00
50db2fb372 [Feature] MultiGen Eval (Backend) (#117)
Fixes #16

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#117
2025-09-18 18:40:45 +12:00
762a6b7baf [Feature] Package Export/Import (#116)
Fixes #90
Fixes #91
Fixes #115
Fixes #104

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#116
2025-09-16 02:41:10 +12:00
ce349a287b [Migration] Migration file for #113 (#114)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#114
2025-09-11 18:23:14 +12:00
7905a8c2c1 [Bug] Fixes for Group handling, Load all objects only if necessary (#113)
Fixes #109
Fixes #110
Fixes #111
Fixes #112

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#113
2025-09-11 18:19:27 +12:00
af3981d96e [Bug] Fix duplicate IDs in SVG defs (#108)
Fixes #107

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#108
2025-09-11 10:20:33 +12:00
62e6448448 [Feature] Legacy Package Import (#106)
Fixes #105

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#106
2025-09-11 01:18:04 +12:00
31783306e2 [Feature] Pathway SVG Export (#102)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#102
2025-09-10 18:44:18 +12:00
e82fe7e87e [Feature] Initial Active Directory / Entra Login (#101)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#101
2025-09-10 08:29:27 +12:00
4463bf1bc8 [Fix] Add RuleBasedRelativeReasoning to localize_urls command (#95)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#95
2025-09-09 21:43:11 +12:00
abfd683640 [Migration] Migration for Rule Based Modelling (#94)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#94
2025-09-09 21:27:10 +12:00
3453a169e1 [Feature] Async Prediction Status Poll (#93)
Fixed #81

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#93
2025-09-09 20:39:26 +12:00
5477b5b3d4 [Feature] Rule Based Model (#92)
Fixes #89

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#92
2025-09-09 19:32:12 +12:00
1a6608287d [Feature] Password Reset Flow (#88)
Fixes #83

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#88
2025-09-06 19:53:36 +12:00
a16035677c [Bugfix] Added Missing Paper in Cite Modal (#87)
Fixes #77

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#87
beta_2025-09-04
2025-09-05 08:21:46 +12:00
498a53ab3d [Feature] Move Login exempt urls to settings (#86)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#86
2025-09-05 08:04:20 +12:00
aa3b53e94b [Bugfix] Login required exempts auth for SSO Flow (#85)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#85
2025-09-05 07:59:14 +12:00
3c8f0e80cb [Feature] OAuth2 Provider (#84)
Fixes #74

Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#84
2025-09-05 06:50:16 +12:00
4158bd36cb [Feature] Legacy API Layer (#80)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#80
beta_2025-09-02
2025-09-03 01:35:51 +12:00
4e02910c62 Add Link to Community in Navbar (#79)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#79
2025-09-02 08:28:37 +12:00
2babe7f7e2 [Feature] Scenario Creation (#78)
Co-authored-by: Tim Lorsbach <tim@lorsba.ch>
Reviewed-on: enviPath/enviPy#78
2025-09-02 08:06:18 +12:00