MalAvi
Avian haemosporidian database

MalAvi · orientation

How MalAvi
works after the update

This document describes the changes in the rebuild of MalAvi. It will show you how the rebuild works in sort of broad strokes. Briefly, there's a new static website with a submission button, curators review the submissions and can add papers themselves, approved submissions get queued up for the next build.

I also describe the other connected products, there's the new website of course, but also the R package malaviR, and there's a BLAST-like sequence matcher on the website (called MalAvi BLAST, like the old tool) that replaces the shinyapp BLAST tool we were using.

Two ways data arrive

New data are incorporated in two ways. Either someone submits new data (the submission form and/or a manuscript), or a curator finds the data in a published paper and submits the data themselves. Once the data are submitted to our pipeline they will get the same checks.

SOMEONE SUBMITS Filled-in template (the excel submission file) + paper, if published A CURATOR FINDS A PAPER PDF and its supplementary tables web form (linked on new MalAvi site) extraction (some programming help possible, but will still be mostly manual) Automated checks once data are submitted Is this name taken? Is this sequence already a named lineage? Any strange sequence patterns or strange geographic or host distribution patterns? One queue (visible on the website) The queue is monitored so that name reservations are preserved by order of submission the curators Decision See next section for details.
Nothing is written to the database without the curators' approval.

The software underlying the pipeline is deterministic everywhere (not AI). There's no risk of AI hallucinated data getting into the database.

How a curator decision is made

Form is live

When a submission happens, the curators are emailed a report (with the data quality checks). At least one curator must read the report and record one of three answers ("Accept", "Flag", "Reject"). The rule that matters most is what happens when two curators disagree: an objection outranks an approval. Currently, I've put a three-day period after one curator approves before the submission is accepted (that time window can change). During that window, another curator can review and either accept as well or flag or reject (again the latter two choices will stop the submission going forward). Curators must discuss among each other any flagged or rejected submission before it can be resurrected and submitted again. Only one curator (currently set as V. Ellis) has the authority to override the flagged or rejected setting.

Report generated from a submission Curators receive the report in an email and one click to review. Curator reads the report and data submission Curator picks accept, flag, or reject Accept Waits three days in case other curators want to look Into the next release Flag Blocked written reason required from curator a flag during those three days from a second curator stops it Lead curator clears the flag (leading to acceptance or rejection), and records the decision and justification Reject Not included written reason required by curator (lead curator can still overrule in case of a discrepancy, one reject and one accept for example).
One approval is enough to move a submission forward, but a flag or rejection by another curator stops the submission. Clearing someone else's flag is a lead curator's action and records who was consulted so that there are records of all the decisions.

How a new version of MalAvi is generated

Being built

When the curators deem enough submissions are in, one of them will run the software that builds the new version. A diff file will be created that shows the differences from one version to another. This also allows curators to make ad hoc fixes that are identified as well.

Accepted records since the last version The database every record from current version Build a new version curator chooses when to run the software the version is dated The new release is built malaviR package The new website MalAvi BLAST (or anything that needs it)
A single, dated, versioned release is produced and can be used in any tools we build. The website allows direct download of the full dataset (malaviR is currently built to host old dataset versions as well).

Who does what

Curators review submitted data and can add data from papers they find, but they don't need to know any coding. Only the maintainer needs to deal with code (and the maintainer is likely also one of the curators, probably the lead curator). Note that submitters will need Google accounts to submit with the machinery I've set up, but if they can't do that, they can just email their submissions to the curators.

Submitter

Fills in and submits the template, and/or a paper. Can follow their submission's progress on the website.

A web form with Google acct, or just send an email

Curator

Reviews the report generated by submission and records a verdict.

Browser and email needed...easy.

Lead curator

Everything a curator does, plus clearing another curator's flag or overriding a rejection after discussion.

Browser and email needed...easy.

Maintainer

Keeps the software running and builds each new version.

Command line programming needed.

Unpublished data can be submitted and are held privately. This is (as it's always been) so someone can reserve a lineage name before their paper is out. The queue on the website shows the name and the date the lineage name was claimed, but does not show who claimed it or anything else about the parasite.