Standard medical LLMs (BioMistral, Llama-3-Med, ChatGPT, Med-PaLM) predict numbers probabilistically. When tasked with clinical equations, pediatric dosages, or multi-drug interactions, they generate plausible-sounding values that drift from clinical guidelines and risk catastrophic toxicity.
MedOrqis-LLM couples a bilingual 6B narrative reasoning brain with a deterministic execution runtime. It achieves 0.00% arithmetic error, 100.0% pediatric overdose safety cap enforcement, and full Arabic regional dialect triage on a sovereign 7.68 GB VRAM footprint.
Head-to-head evaluation across 5 foundational model paradigms on clinical safety, mathematical precision, linguistic triage, and deployment sovereignty.
| Evaluation Dimension |
MedOrqis-LLM (6B QLoRA + Tool Engine) |
Karnak-6B (Raw Unfine-Tuned Base) |
BioMistral-7B (Open Medical Specialist) |
Llama-3-8B-Inst (Open General Foundation) |
GPT-4o / Claude 3.5 (Commercial Frontier Cloud) |
|---|---|---|---|---|---|
|
Arithmetic Error Rate
CKD-EPI 2021, Schwartz, CrCl equations
|
0.00% Error Deterministic Tool Grounding |
29.3% - 41.5% Drift Floating-point hallucination |
24.8% Drift Probabilistic guess in weights |
18.4% Drift Approximates exponent math |
8.2% - 15.4% Error Without internal code sandbox |
|
Pediatric Overdose Safety Cap
Weight-based dose exceeding adult ceiling
|
100.0% Enforced Mandatory adult single-dose ceiling |
Failed (0.0%) Dispenses raw unconstrained dose |
35.0% Adherence Calculates 450mg for 400mg cap |
50.0% Adherence Inconsistent warning in text |
78.0% Adherence Warns verbally, math still raw |
|
Structured Tool Calling Syntax
Native ChatML <tool_call> schema
|
100.0% Valid Syntax Zero JSON parsing failure |
0.0% (Unsupported) Generates unstructured text |
0.0% (Unsupported) No function calling training |
62.5% Adherence Generic schema without tools |
98.0% Adherence Requires complex API wrappers |
|
Arabic Regional Dialect Triage
Egyptian, Saudi, Levantine colloquialisms
|
98.0% Comprehension Maps colloquial terms to ICD-10 |
58.0% Comprehension Fails on regional drug brands |
18.0% Comprehension English/French trained bias |
42.0% Comprehension Translates literally to English |
86.0% Comprehension Lacks Middle East brand catalog |
|
Trade-to-Generic Pharmacology
Mapping Concor, Plavix, Cataflam to INN
|
100.0% Mapped Active molecule extraction |
45.0% Accuracy Recognizes generic only |
22.0% Accuracy Western EU trade bias |
54.0% Accuracy US FDA brand bias |
79.0% Accuracy Good general, misses local brands |
|
Fatal Drug-Drug Interaction Check
Nitrates + PDE5i, Statins + Macrolides
|
100.0% Blocked Immediate Red-Flag Escalation |
60.0% Detection Narrative warning buried in text |
68.0% Detection Identifies interaction weakly |
72.0% Detection Soft caveat without hard refusal |
94.0% Detection Strong verbal safety warning |
|
Guideline Version Entailment
ESC 2024 sex-neutral, KDIGO 2024, Sepsis-3
|
Strict Guideline Pinning Separates ESC 2024 from 2010 |
Confabulated Mixes outdated formulas |
Outdated (Pre-2022) Uses race-adjusted eGFR |
Inconsistent Varies between guideline years |
Variable Requires explicit year in prompt |
|
Hospital Sovereign Data Privacy
HIPAA, GDPR, National Health Cloud Residency
|
100% On-Premise Air-Gap Zero external network transmission |
Air-Gapped Capable Open-weights local hosting |
Air-Gapped Capable Open-weights local hosting |
Air-Gapped Capable Open-weights local hosting |
Violates Air-Gap Transmits patient PHI to cloud |
|
Inference Hardware Footprint
VRAM requirement and minimum GPU tier
|
7.68 GB VRAM Single RTX 3060 / T4 / L4 |
14.2 GB (16-bit) / 7.5 GB Commodity GPU |
15.8 GB (16-bit) / 8.2 GB Commodity GPU |
16.5 GB (16-bit) / 9.0 GB Commodity GPU |
External Cluster Proprietary multi-GPU datacenter |
Inspect the exact outputs generated by each model on real, safety-critical medical scenarios. Select a case below to analyze the differential.
Patients in acute clinical distress communicate in regional vernacular rather than Modern Standard Arabic (Fusha). MedOrqis-LLM resolves Egyptian, Saudi/Gulf, Levantine, and Maghrebi idioms into standardized clinical ontologies (SNOMED-CT / ICD-10).
Autoregressive models are statistically trained to generate plausible token sequences, not evaluate non-linear mathematical formulas. MedOrqis enforces architectural decoupling.
Handled inside the 6B transformer weights via QLoRA SFT. The model excels at linguistic comprehension, clinical synthesis, and parsing unstructured dialectal terminology into standardized nomenclature.
Mathematical calculations, multi-drug interaction graphs, and risk scoring are strictly prohibited inside the neural weights. The model emits a structured function call targeting validated Python calculators.
Deploy MedOrqis-LLM locally on commodity hospital infrastructure with zero external telemetry.
vllm serve Applied-Innovation-Center/Karnak-6B-v1.0 \
--enable-lora \
--lora-modules medorqis-llm=seotarek/medorqis-llm \
--max-model-len 4096 \
--gpu-memory-utilization 0.90 \
--port 8000
By keeping the base weights in 4-bit / 16-bit and mounting the 198MB adapter dynamically, clinical institutions can serve multiple specialized medical models on commodity on-premise GPUs.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "Applied-Innovation-Center/Karnak-6B-v1.0"
lora_id = "seotarek/medorqis-llm"
tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(base_id, load_in_4bit=True, device_map="auto")
model = PeftModel.from_pretrained(base_model, lora_id)
prompt = tokenizer.apply_chat_template(messages, tools=CLINICAL_TOOLS, tokenize=False)
Passing the clinical tools schema to apply_chat_template triggers valid ChatML tool calling representations.