Final project · AI / ML

Hi, I'm Pranav Chaturvedi.

This deck explains my final project, OmniFeedback AI, through its Python code: from raw data to a GenAI copilot.

Use ← → to move between slides. Every code panel links to the exact lines on GitHub, and every concept chip opens the theory sheet.

Open the theory sheet Source on GitHub ↗

From JaipurRajasthan, India
B.Tech in Computer ScienceVIT Vellore
Working at LumberFiBengaluru · construction-payroll software
Took a career break
Built OmniFeedback AI end to endData → database → ML & deep learning → GenAI → dashboard

THE PROJECT

The problem, and the solution I applied

A feedback triage platform, built so that its own metrics can be trusted.

The problem

Too much feedback, too many channels. Twitter, email, reviews, call notes and in-app comments pile up faster than anyone can read them.
Urgent complaints get buried. A “double charged, fix this now!” looks like any other ticket until someone reads it.
Metrics can lie. My first model scored a perfect 1.0, but it had only memorised 9 sentences.

What I built

Triage: an urgency score from 0 to 1 for every item.
Themes: unsupervised clustering of what customers talk about.
Resolution: a GenAI copilot that drafts root cause, action and a reply, plus entity extraction and executive summaries.

Same pipeline, three versions: the fix made visible

V1 · NAIVE

1.0

F1 and silhouette are perfect, but there are only 9 unique sentences. Memorisation.

V2 · HARDENED

0.33

Varied, noisy data gives honest scores. But its own 196-word vocabulary fails on real wording: separation 0.02.

V3 · TRANSFER

0.30

A pretrained DistilBERT separates critical from positive about 13× better, on the same data.

The ideaDon't just report a good number. Show why a number can be misleading, then fix the cause (data first, model second), and prove it on real text.

Python 3.11SQLitescikit-learnPyTorchHugging FaceClaude APIStreamlitpytestDocker

THE FRAMEWORK

The pipeline, and how each slide reads

Five stages. Each slide shows real code, what it does, and the concept behind it.

1 · DATA

Generate & clean

generate_data.py
ingestor.py

Slides 4 – 5

2 · DB

Store safely

database.py
star schema
ACID transactions

Slides 6 – 8

3 · TRAIN

Learn

ml_models.py
dl_lstm.py
transformer_regressor.py

Slides 9 – 11

4 · EVAL

Judge honestly

F1, MSE, silhouette
stress & real-data tests

Slides 12 – 13

5 · APP

Use it

GenAI copilot, NER
Streamlit dashboard
tests, Docker

Slides 14 – 15

How to read a slide

Code panel: real lines from the repo. The pill at the top right is a GitHub permalink with the line numbers, pinned to commit d27a741 so it never drifts.
Highlighted lines: the ones that carry the idea.
◆ INHERITANCE where a class inherits from a parent (the four PyTorch classes).
● SHARED FUNCTION a function or object used from many places (cleaner, variant registry, dispatcher).
PYTHON FEATURE language features worth knowing: decorators, closures, comprehensions.
Concept chips open the matching entry on the theory sheet.

Reproduce it (from the README)

terminalREADME
1python scripts/generate_data.py                      # V1 + V2 datasets
2python scripts/run_pipeline.py v1_naive v2_hardened  # clean → DB → train
3python scripts/train_v3.py                           # fine-tune DistilBERT
4python scripts/stress_test_compare.py                # V2 vs V3 evidence
5streamlit run app.py                                 # the dashboard

The dashboard and models ship pre-built, so the live app needs no training step.

Stage 1 of 5 · DATA → DB → TRAIN → EVAL → APP

Generating data: the V1 trap and the V2 fix

There is no real customer dataset, so I wrote two generators. Same schema, very different difficulty.

scripts/generate_data.pyscripts/generate_data.py · L44–69
44def generate_naive(num_records: int = 4000, seed: int = 42) -> pd.DataFrame:
  """Verbatim-phrase generator: same structure/rate as the original capstone notebook."""
  rng = random.Random(seed)
  base_time = datetime.now() - timedelta(days=30)
  records = []
  for i in range(1, num_records + 1):
      is_negative = rng.random() < 0.4
      if is_negative:
          text = rng.choice(NAIVE_NEGATIVE_PHRASES)
          urgency_score = round(rng.uniform(0.6, 1.0), 4)
          label = 0
      else:
          text = rng.choice(NAIVE_POSITIVE_PHRASES)
          urgency_score = round(rng.uniform(0.0, 0.4), 4)
          label = 1
      records.append({
          "feedback_id": 10000 + i,
          "user_id": rng.randint(100, 999),
          "raw_text": text,
          "channel": rng.choice(CHANNELS),
          "aspect_category": rng.choice(ASPECTS),
          "urgency_score": urgency_score,
          "label": label,
          "timestamp": (base_time + timedelta(minutes=rng.randint(0, 43200))).strftime("%Y-%m-%d %H:%M:%S"),
      })
  return pd.DataFrame(records)

scripts/generate_data.pyscripts/generate_data.py · L71–139
71CRITICAL_TEMPLATES = [
  "App keeps crashing during {step} step, this is unacceptable!",
  "Double charged on my {payment} for {product}, fix this immediately!",
  "Package arrived {condition} and support is completely ignoring my emails.",
  "System latency is terrible after the {event}, nothing works.",
  "Major outage across the platform, unable to {action} for hours.",
  "Your {product} deleted all my data without warning, I need this fixed NOW.",
  "Been on hold for {duration} trying to reach a human, this is ridiculous.",
  "The {feature} is completely broken since the last update, losing customers.",
80]
81CRITICAL_FILLS = {
  "step": ["checkout", "payment", "login", "upload"],
  "payment": ["credit card", "debit card", "PayPal account"],
  "product": ["order", "subscription", "account", "invoice"],
  "condition": ["damaged", "empty", "the wrong item"],
  "event": ["latest update", "server migration", "new release"],
  "action": ["log in", "check out", "reach support"],
  "duration": ["45 minutes", "an hour", "two hours"],
  "feature": ["search bar", "notifications", "export tool", "dashboard"],
90}
⋯
135def _fill(template: str, fills: dict) -> str:
  for key, options in fills.items():
      if f"{{{key}}}" in template:
          template = template.replace(f"{{{key}}}", random.choice(options), 1)
  return template

scripts/generate_data.pyscripts/generate_data.py · L151–175
151roll = random.random()
152if roll < critical_rate:
153    text = _fill(random.choice(CRITICAL_TEMPLATES), CRITICAL_FILLS)
154    urgency_score = round(random.uniform(0.65, 1.0), 4)
155    label = 0
156elif roll < critical_rate + neutral_rate:
157    text = _fill(random.choice(NEUTRAL_TEMPLATES), NEUTRAL_FILLS)
158    urgency_score = round(random.uniform(0.35, 0.6), 4)
159    label = 1
160elif roll < critical_rate + neutral_rate + ambiguous_rate:
161    text = _fill(random.choice(AMBIGUOUS_TEMPLATES), AMBIGUOUS_FILLS)
162    # ambiguous cases straddle the escalation threshold on purpose
163    urgency_score = round(random.uniform(0.4, 0.7), 4)
164    label = 0 if urgency_score >= 0.5 else 1
165else:
166    text = _fill(random.choice(POSITIVE_TEMPLATES), POSITIVE_FILLS)
167    urgency_score = round(random.uniform(0.0, 0.3), 4)
168    label = 1
⋯
170# Small fraction of realistic label noise (mislabeled/annotator disagreement),
171# so downstream models can't trivially hit a perfect score on this dataset.
172if random.random() < label_noise_rate:
173    label = 1 - label
174    urgency_score = round(1.0 - urgency_score, 4)
175 

	V1 naive	V2 hardened
Text source	9 fixed sentences	templates + random fills
Unique texts	9	212
Critical rate	40%	15%
Ambiguous cases	none	15% straddling 0.5
Label noise	none	3%
Baseline macro-F1	1.00	0.91

What the code does

generate_naive picks from a short list: easy to separate, and the model just memorises it.
_fill swaps each {placeholder} for a random choice. One template becomes many sentences.
Ambiguous rows mix praise and complaint, with scores in 0.4 to 0.7. Then 3% of labels are flipped on purpose.

default args + type hintslist of dicts → DataFrameprivate RNG vs global seed

Why it mattersA perfect score on V1 is a symptom, not a success. Changing the data, not the code, made the metrics honest.

Theory sheet →Synthetic data & random seeds Template-based text generation Label noise Overfitting & memorisation Type hints & default arguments

Stage 1 of 5 · DATA → DB → TRAIN → EVAL → APP

Cleaning: one shared function, called everywhere

Regular expressions turn messy text into a consistent form. The same function cleans training data and live user input.

src/ingestor.pysrc/ingestor.py · L6–39
6_HTML_TAG_RE = re.compile(r"<[^>]+>")
7_URL_RE = re.compile(r"https?://\S+|www\.\S+")
8_HASHTAG_RE = re.compile(r"#(\w+)")
9_MENTION_RE = re.compile(r"@(\w+)")
10_NON_ALNUM_RE = re.compile(r"[^a-z0-9\s']")
11_WHITESPACE_RE = re.compile(r"\s+")
⋯
26@classmethod
27def clean_text(cls, text: str) -> str:
28    """Strip HTML/URLs/mentions/special chars, collapse repeats & whitespace, lowercase."""
29    if not isinstance(text, str) or not text.strip():
30        return ""
31    t = text.lower()
32    t = cls._HTML_TAG_RE.sub(" ", t)
33    t = cls._URL_RE.sub(" ", t)
34    t = cls._MENTION_RE.sub(" ", t)
35    t = cls._HASHTAG_RE.sub(r"\1", t)  # keep hashtag word, drop the '#'
36    t = cls._REPEAT_CHAR_RE.sub(r"\1\1", t)  # "sooooo" -> "soo"
37    t = cls._NON_ALNUM_RE.sub(" ", t)
38    t = cls._WHITESPACE_RE.sub(" ", t).strip()
39    return t

◆ NO INHERITANCE: plain class@classmethod + class attributescompiled regexguard clause

A real warehouse row

raw: Love the new checkout flow update! Very intuitive.
clean: love the new checkout flow update very intuitive

Design choiceThe BiLSTM and TF-IDF get cleaned text. DistilBERT gets raw text, because its tokenizer uses casing and punctuation (“!!!”) as signal. The dispatcher on slide 8 enforces that rule.

Replaying `clean_text` on a messy input (real class, real output)

input: <b>SOOOO</b> bad!!! @support the app is down https://x.co/a #outage

lower-case
<b>soooo</b> bad!!! @support the app is down https://x.co/a #outage
strip HTML tags
soooo bad!!! @support the app is down https://x.co/a #outage
strip URLs
soooo bad!!! @support the app is down #outage
strip @mentions
soooo bad!!! the app is down #outage
keep #hashtag word
soooo bad!!! the app is down outage
shrink repeats
soo bad!! the app is down outage
drop punctuation
soo bad the app is down outage
collapse spaces
soo bad the app is down outage

● SHARED FUNCTION called from 7 places (outside tests)

1_Inference_Playground.py · L503_GenAI_Copilot.py · L46run_pipeline.py · L33stress_test_compare.py · L35superiority_scorecard.py · L49train_v3.py · L35validate_real_data.py · L36

Theory sheet →Regular expressions Text normalisation Classes, class attributes & @classmethod Guard clauses & early return pandas: DataFrame, masks, apply, groupby

Stage 2 of 5 · DATA → DB → TRAIN → EVAL → APP

Star schema: one fact table, small dimension tables

Press Play to watch one record travel into the warehouse and back out through a JOIN.

⬇ incoming feedback_id=11670 · user_id=871 · channel="Twitter" · aspect="General Praise" · score=0.05

Dim_User (user_id)

479user

655user

871user

⋯886 users in total

Dim_Channel (id, name)

1Call Center Note

2Twitter

3Mobile App

4Web Review

5Email Support

Dim_Product (id, aspect)

1Billing Defect

2General Praise

3App Crash

4Product Quality

5Shipping Delay

6UI Ergonomics

Fact_Feedback (feedback_id, user_id, channel_id, aspect_id, texts, score, …)

⋯other rows

11670user=871 channel=2 aspect=2 · "Love the new dashboard update! Ve…" · 0.05

JOIN11670 · Twitter · General Praise · "Love the new dashboard update! Ve…" · 0.05

src/database.pysrc/database.py · L15–42
15CREATE TABLE IF NOT EXISTS Dim_Channel (
16    channel_id INTEGER PRIMARY KEY AUTOINCREMENT,
17    channel_name TEXT NOT NULL UNIQUE
18);
⋯
25CREATE TABLE IF NOT EXISTS Fact_Feedback (
26    feedback_id INTEGER PRIMARY KEY,
27    user_id INTEGER NOT NULL,
28    channel_id INTEGER NOT NULL,
29    aspect_id INTEGER NOT NULL,
30    raw_text TEXT NOT NULL,
31    cleaned_text TEXT NOT NULL,
32    urgency_score REAL NOT NULL CHECK (urgency_score >= 0.0 AND urgency_score <= 1.0),
33    label INTEGER,
34    predicted_cluster INTEGER,
35    timestamp TEXT NOT NULL,
36    FOREIGN KEY (user_id) REFERENCES Dim_User(user_id),
37    FOREIGN KEY (channel_id) REFERENCES Dim_Channel(channel_id),
38    FOREIGN KEY (aspect_id) REFERENCES Dim_Product(aspect_id)
39);
⋯
41CREATE INDEX IF NOT EXISTS idx_fact_timestamp ON Fact_Feedback(timestamp);
42CREATE INDEX IF NOT EXISTS idx_fact_channel ON Fact_Feedback(channel_id);

src/database.pysrc/database.py · L66–72
66def _get_or_create_dim(self, conn, table: str, name_col: str, value: str) -> int:
  cur = conn.execute(f"SELECT rowid FROM {table} WHERE {name_col} = ?", (value,))
  row = cur.fetchone()
  if row:
      return row[0]
  cur = conn.execute(f"INSERT INTO {table} ({name_col}) VALUES (?)", (value,))
  return cur.lastrowid

src/database.pysrc/database.py · L125–136
125def fetch_all_feedback(self) -> pd.DataFrame:
  return self.query(
      """
      SELECT f.feedback_id, f.user_id, c.channel_name AS channel,
             p.aspect_category, f.raw_text, f.cleaned_text,
             f.urgency_score, f.label, f.predicted_cluster, f.timestamp
      FROM Fact_Feedback f
      JOIN Dim_Channel c ON f.channel_id = c.channel_id
      JOIN Dim_Product p ON f.aspect_id = p.aspect_id
      ORDER BY f.timestamp
      """
  )

Fact = the events (feedback). Dimensions = the labels (user, channel, aspect), each stored once.
In the V2 warehouse: 5 channels, 6 aspects, 886 users for 4,000 facts.
PRIMARY KEY, FOREIGN KEY, UNIQUE and CHECK enforce the rules inside the database.

@contextmanagerparameterised SQL (?)● WarehouseManager: used by 6 files

Theory sheet →Star schema JOIN Primary & foreign keys Constraints Database index Idempotent writes Parameterised queries & SQL injection

Stage 2 of 5 · DATA → DB → TRAIN → EVAL → APP

ACID: a batch either lands completely or not at all

Switch between a bad and a good batch, then press Play.

BEGIN

INSERT row 1 · score 0.52 → pending

INSERT row 2 · score 0.31 → pending

✗ INSERT row 3 · score 5.0 → CHECK constraint failed

ROLLBACK → rows 1 and 2 undone

Fact_Feedback (what the dashboard can see)

∅empty

1score 0.52 · pending

2score 0.31 · pending

∅0 rows · same as before the batch

BEGIN

INSERT row 1 · score 0.52 → pending

INSERT row 2 · score 0.31 → pending

INSERT row 3 · score 0.88 → pending

COMMIT → 3 rows saved

Fact_Feedback (what the dashboard can see)

∅empty

1score 0.52

2score 0.31

3score 0.88

src/database.pysrc/database.py · L89–119
89inserted = 0
90with self.get_connection() as conn:
91    try:
92        conn.execute("BEGIN")
93        for _, row in df.iterrows():
⋯
114            inserted += 1
115        conn.commit()
116    except Exception:
117        conn.rollback()
118        raise
119return inserted

tests/test_database.pytests/test_database.py · L13–53
13@pytest.fixture
14def db():
15    with tempfile.TemporaryDirectory() as tmp:
16        yield WarehouseManager(db_path=Path(tmp) / "test.db")
⋯
47def test_rollback_on_bad_urgency_score(db):
48    df = _sample_df(2)
49    df.loc[1, "urgency_score"] = 5.0  # violates CHECK constraint (0.0-1.0)
50    with pytest.raises(Exception):
51        db.insert_feedback_batch(df)
52    result = db.fetch_all_feedback()
53    assert len(result) == 0  # entire batch rolled back, nothing partially committed

Letter	Guarantee	Where in this project
A	all or nothing	src/database.py · L92 + src/database.py · L117
C	rules always hold	src/database.py · L32
I	no half-seen work	SQLite's own locking (no project code)
D	survives a crash	src/database.py · L115 writes to the file

try / except / raise@contextmanager → withpytest.raises

Why it mattersA half-loaded batch can never reach the dashboard. One transaction for all 4,000 rows is also faster than committing each row, because every commit forces a disk write.

Theory sheet →ACID Transaction, commit & rollback Constraints Context managers pytest: fixtures, monkeypatch & mocks

CODE ARCHITECTURE

Inheritance and shared functions: how the code is organised

Four classes inherit from PyTorch; a few global functions are called from many places so that logic exists only once.

src/variants.pysrc/variants.py · L42–55
42"v3_transformer": {
⋯
51    "raw_csv": ROOT / "data" / "raw" / "omnifeedback_v2_hardened.csv",
52    "db_path": ROOT / "data" / "warehouse" / "omnifeedback_v2_hardened.db",
53    "model_dir": ROOT / "models" / "v3_transformer",
54    "model_type": "transformer",
55},

src/urgency.pysrc/urgency.py · L9–23
9def load_urgency_model(variant: dict):
  if variant.get("model_type") == "transformer":
      model, tokenizer, device, max_len = load_transformer_regressor(model_dir=variant["model_dir"])
      return {"type": "transformer", "model": model, "tokenizer": tokenizer, "device": device, "max_len": max_len}
  model, vocab, device = load_bilstm_model(model_dir=variant["model_dir"])
  return {"type": "bilstm", "model": model, "vocab": vocab, "device": device}
⋯
17def score_urgency(bundle: dict, text: str, cleaned_text: str = None) -> float:
  """`cleaned_text` (regex-cleaned) is used for BiLSTM; transformer models score
  the raw text directly since their pretrained tokenizer benefits from real
  punctuation/casing that the BiLSTM's cleaning pipeline strips out."""
  if bundle["type"] == "transformer":
      return predict_urgency_transformer(bundle["model"], bundle["tokenizer"], text, bundle["device"], bundle["max_len"])
  return predict_urgency(bundle["model"], bundle["vocab"], cleaned_text or text, bundle["device"])

src/app_state.pysrc/app_state.py · L11–27
11def select_variant() -> dict:
  if "variant_id" not in st.session_state:
      st.session_state["variant_id"] = DEFAULT_VARIANT

  with st.sidebar:
      st.markdown("### 🔀 Dataset / Model Version")
      variant_id = st.radio(
          "Compare naive vs. hardened training",
          options=list(VARIANTS.keys()),
          format_func=lambda v: VARIANTS[v]["short_label"],
          index=list(VARIANTS.keys()).index(st.session_state["variant_id"]),
          key="variant_id",
      )
      st.caption(VARIANTS[variant_id]["description"])
      st.divider()

  return VARIANTS[variant_id]

◆ INHERITANCE four classes

Class & code	Parent	What the child gets for free
`FeedbackDataset` src/dl_lstm.py · L45–56	`Dataset`	__len__, __getitem__ → works with DataLoader
`BiLSTMUrgencyRegressor` src/dl_lstm.py · L59–73	`nn.Module`	model(x) → forward(x), .parameters(), .train()/.eval(), .state_dict()
`UrgencyTextDataset` src/transformer_regressor.py · L31–44	`Dataset`	same contract, returns a dict
`TransformerUrgencyRegressor` src/transformer_regressor.py · L47–81	`nn.Module`	same machinery + frozen DistilBERT inside

super().__init__() is mandatory: without it PyTorch would not register the layers. Not inherited, by design: TextIngestor, WarehouseManager, TextVocabulary.

● SHARED FUNCTION global / shared functions

Function	Used by	Why
`TextIngestor.clean_text`	7 files	cleans text everywhere
`VARIANTS`	11 files	single source of paths and model types
`select_variant()`	6 files	shared sidebar toggle
`load_urgency_model / score_urgency`	6 files	one entry point to either model
`WarehouseManager(...)`	6 files	all database access

V3 in one line of configV3 reuses V2's db_path and raw_csv but has its own model_dir and model_type = "transformer". The dispatcher branches on that tag, so no page had to change.

Rule enforced in one placescore_urgency sends cleaned text to the BiLSTM and raw text to DistilBERT. Every page and script gets this for free.

Theory sheet →Inheritance & super()Registry & dispatcher pattern Classes, class attributes & @classmethod Decorators Streamlit

Stage 3 of 5 · DATA → DB → TRAIN → EVAL → APP

Classical ML: TF-IDF, K-Means and the Kneedle elbow

Text becomes numbers (TF-IDF), numbers become topics (K-Means), and the Kneedle rule picks how many topics, without being fooled.

src/ml_models.pysrc/ml_models.py · L50–70
50ks = sorted(k_scores.keys())
51scores = np.array([k_scores[k]["silhouette"] for k in ks], dtype=float)
52xs = np.array(ks, dtype=float)
53 
54xs_norm = (xs - xs.min()) / (xs.max() - xs.min())
55ys_norm = (scores - scores.min()) / (scores.max() - scores.min() + 1e-12)
56 
57x1, y1 = xs_norm[0], ys_norm[0]
58x2, y2 = xs_norm[-1], ys_norm[-1]
59line_vec = np.array([x2 - x1, y2 - y1])
60line_len = np.linalg.norm(line_vec)
61if line_len == 0:
62    return ks[0]
63 
64distances = []
65for x, y in zip(xs_norm, ys_norm):
66    point_vec = np.array([x - x1, y - y1])
67    cross = line_vec[0] * point_vec[1] - line_vec[1] * point_vec[0]
68    distances.append(abs(cross) / line_len)
69 
70return ks[int(np.argmax(distances))]

src/ml_models.pysrc/ml_models.py · L17–33
17def fit_tfidf(texts: list, max_features: int = 2000) -> TfidfVectorizer:
  vectorizer = TfidfVectorizer(max_features=max_features, ngram_range=(1, 2), min_df=2)
  vectorizer.fit(texts)
  return vectorizer
⋯
23def select_k_via_silhouette(X, k_range=range(2, 13), random_state: int = 42) -> dict:
  """Elbow/silhouette sweep to justify the chosen K instead of guessing."""
  scores = {}
  for k in k_range:
      km = KMeans(n_clusters=k, random_state=random_state, n_init=10)
      labels = km.fit_predict(X)
      scores[k] = {
          "inertia": float(km.inertia_),
          "silhouette": float(silhouette_score(X, labels)) if k > 1 else 0.0,
      }
  return scores

V1 naiveV2 hardened

Kneedle on the project's own test case

k	silhouette	normalised	distance from chord
2	0.10	0.00	0.00
3	0.30	0.87	0.38 ← max
4	0.32	0.96	0.20
5	0.33	1.00	0.00

Normalise both axes, draw a straight line from first to last point, pick the point farthest from it: here k = 3.
argmax(silhouette) would always pick the biggest k on templated text. Two unit tests lock the fix in.

numpy vector mathszip() loopearly return

Theory sheet →TF-IDF K-Means clustering Silhouette score Elbow method & Kneedle PCA Model serialisation

Stage 3 of 5 · DATA → DB → TRAIN → EVAL → APP

Deep learning: the BiLSTM, written from scratch in PyTorch

This is where inheritance matters: a class that extends nn.Module gets the whole training machinery for free.

src/dl_lstm.pysrc/dl_lstm.py · L59–73
59class BiLSTMUrgencyRegressor(nn.Module):
  def __init__(self, vocab_size: int, embed_dim: int = 128, hidden_dim: int = 64, dropout: float = 0.3):
      super().__init__()
      self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
      self.lstm = nn.LSTM(embed_dim, hidden_dim, batch_first=True, bidirectional=True)
      self.dropout = nn.Dropout(dropout)
      self.fc = nn.Linear(hidden_dim * 2, 1)
      self.sigmoid = nn.Sigmoid()

  def forward(self, x):
      embedded = self.embedding(x)
      _, (h_n, _) = self.lstm(embedded)
      h_cat = torch.cat((h_n[-2], h_n[-1]), dim=1)  # last fwd + bwd hidden states
      out = self.dropout(h_cat)
      return self.sigmoid(self.fc(out)).squeeze(-1)

src/dl_lstm.pysrc/dl_lstm.py · L45–56
45class FeedbackDataset(Dataset):
  def __init__(self, texts: list, scores: list, vocab: TextVocabulary, max_len: int = 128):
      self.encoded = [vocab.encode(t, max_len) for t in texts]
      self.scores = scores

  def __len__(self):
      return len(self.encoded)

  def __getitem__(self, idx):
      x = torch.tensor(self.encoded[idx], dtype=torch.long)
      y = torch.tensor(self.scores[idx], dtype=torch.float32)
      return x, y

src/dl_lstm.pysrc/dl_lstm.py · L119–135
119model.train()
120train_losses = []
121for x, y in train_loader:
122    x, y = x.to(device), y.to(device)
123    optimizer.zero_grad()
124    preds = model(x)
125    loss = criterion(preds, y)
126    loss.backward()
127    optimizer.step()
128    train_losses.append(loss.item())
⋯
130model.eval()
131val_losses = []
132with torch.no_grad():
133    for x, y in val_loader:
134        x, y = x.to(device), y.to(device)
135        val_losses.append(criterion(model(x), y).item())

src/dl_lstm.pysrc/dl_lstm.py · L142–151
  if val_loss < best_val_loss - 1e-4:
      best_val_loss = val_loss
      best_state = {k: v.clone() for k, v in model.state_dict().items()}
      epochs_no_improve = 0
  else:
      epochs_no_improve += 1
      if epochs_no_improve >= patience:
          break
⋯
151model.load_state_dict(best_state)

◆ class X(nn.Module)◆ class Y(Dataset)closure: make_loaderdict comprehension

How it flows

Vocabulary & encode
cleaned text → ids, padded to 128 (196-word closed vocabulary)
Embedding → BiLSTM
read forwards and backwards; join last states (64 + 64 = 128)
Dropout → Linear → Sigmoid
one number between 0 and 1 = urgency
MSE loss, Adam, early stopping
stopped at epoch 6 (V1) and 6 (V2)

V1 BiLSTMV2 BiLSTMV3 DistilBERT

Why it fell shortUnseen words become <UNK>, so real customer sentences look alike to V2. That limit is what slide 11 addresses.

Theory sheet →BiLSTM LSTM Inheritance & super()Embedding, vocabulary & out-of-vocabulary words Dataset & DataLoader Dropout Sigmoid MSE loss & regression Optimisers: Adam, AdamW & weight decay Backpropagation & the training loop Early stopping

Stage 3 of 5 · DATA → DB → TRAIN → EVAL → APP

V3: transfer learning with a pretrained DistilBERT

Same data, same split, same task. Only the model changes: it already knows English before it sees my data.

src/transformer_regressor.pysrc/transformer_regressor.py · L47–61
47class TransformerUrgencyRegressor(nn.Module):
  def __init__(self, base_model_name=BASE_MODEL_NAME, n_frozen_layers=N_FROZEN_LAYERS, dropout=0.2):
      super().__init__()
      from transformers import AutoModel
      self.encoder = AutoModel.from_pretrained(base_model_name)

      for param in self.encoder.embeddings.parameters():
          param.requires_grad = False
      for layer in self.encoder.transformer.layer[:n_frozen_layers]:
          for param in layer.parameters():
              param.requires_grad = False

      hidden_size = self.encoder.config.dim
      self.dropout = nn.Dropout(dropout)
      self.head = nn.Linear(hidden_size, 1)

src/transformer_regressor.pysrc/transformer_regressor.py · L63–68
63def forward(self, input_ids, attention_mask):
  out = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
  # mean-pool over non-padding tokens (more robust than [CLS]-only for short, varied text)
  mask = attention_mask.unsqueeze(-1).float()
  pooled = (out.last_hidden_state * mask).sum(1) / mask.sum(1).clamp(min=1e-9)
  return torch.sigmoid(self.head(self.dropout(pooled))).squeeze(-1)

src/transformer_regressor.pysrc/transformer_regressor.py · L75–81
75def trainable_state_dict(self):
  """Only the fine-tuned subset (last N layers + head) — the frozen pretrained backbone
  is re-downloaded from HuggingFace at load time instead of being persisted, since it's
  never modified. Keeps the saved checkpoint ~55MB instead of ~265MB (the full model would
  exceed GitHub's 100MB per-file push limit and require Git LFS for no benefit)."""
  trainable_names = {name for name, p in self.named_parameters() if p.requires_grad}
  return {k: v for k, v in self.state_dict().items() if k in trainable_names}

src/transformer_regressor.pysrc/transformer_regressor.py · L89–96
89"""Same train/val/test split methodology (seed, fractions) as dl_lstm.train_bilstm_model,
90so the held-out test rows are identical and V2 vs V3 metrics are directly comparable.
91 
92Trains on `raw_text` rather than V2's heavily regex-cleaned `cleaned_text` by design:
93DistilBERT's subword tokenizer and pretrained knowledge benefit from real punctuation and
94casing (e.g. "!!!", capitalization) as signal, whereas V2's from-scratch small-vocab BiLSTM
95needed that noise stripped out. Different architectures, different right preprocessing.
96"""

21.4%

parameters trained

0.30

critical − low (V2: 0.02)

0.825

test macro-F1 (V2: 0.821)

	V2 BiLSTM	V3 DistilBERT
Vocabulary	196 words, closed	sub-words, open
Input text	cleaned	raw
Optimiser, lr	Adam, 1e-3	AdamW, 2e-5
Frozen	nothing	embeddings + 4 of 6 layers
Saved checkpoint	all weights	trainable part only

◆ class X(nn.Module)lazy import of transformersset comprehension

requires_grad = False freezes a layer. The optimiser is given only the trainable parameters.
Mean pooling averages token vectors using the attention mask, so padding is ignored.
Test MSE and F1 barely move versus V2 (0.0160 vs 0.0162). The gain shows up on real text, next slides.

Theory sheet →Transfer learning, fine-tuning & freezing BERT & DistilBERT Transformer & self-attention Sub-word tokenisation & the attention mask Mean pooling state_dict & lightweight checkpoints Optimisers: Adam, AdamW & weight decay Lazy imports

Stage 4 of 5 · DATA → DB → TRAIN → EVAL → APP

Evaluation: macro-F1, MSE and silhouette, read honestly

The urgency score is thresholded at 0.5 to get classes, then scored with macro-F1. A perfect number is a warning, not a win.

src/dl_lstm.pysrc/dl_lstm.py · L165–168
165binary_preds = [0 if p >= 0.5 else 1 for p in test_preds]
166binary_targets = [0 if t >= 0.5 else 1 for t in test_targets]
167from sklearn.metrics import f1_score
168test_f1 = float(f1_score(binary_targets, binary_preds, average="macro")) if test_targets else None

src/ml_models.pysrc/ml_models.py · L79–93
79def train_baseline_classifiers(X_train, y_train, X_test, y_test) -> dict:
  """Train LogReg + RandomForest baselines and return metrics + fitted models."""
  results = {}
  for name, clf in [
      ("logistic_regression", LogisticRegression(max_iter=1000, class_weight="balanced")),
      ("random_forest", RandomForestClassifier(n_estimators=200, max_depth=12, class_weight="balanced", random_state=42)),
  ]:
      clf.fit(X_train, y_train)
      preds = clf.predict(X_test)
      results[name] = {
          "model": clf,
          "f1": float(f1_score(y_test, preds, average="macro")),
          "report": classification_report(y_test, preds, output_dict=True),
      }
  return results

src/ml_models.pysrc/ml_models.py · L23–33
23def select_k_via_silhouette(X, k_range=range(2, 13), random_state: int = 42) -> dict:
  """Elbow/silhouette sweep to justify the chosen K instead of guessing."""
  scores = {}
  for k in k_range:
      km = KMeans(n_clusters=k, random_state=random_state, n_init=10)
      labels = km.fit_predict(X)
      scores[k] = {
          "inertia": float(km.inertia_),
          "silhouette": float(silhouette_score(X, labels)) if k > 1 else 0.0,
      }
  return scores

How macro-F1 is computed (V2, all 4,000 rows, in-sample)

	predicted critical	predicted fine
truly critical	1070	275
truly fine	355	2300

critical: precision 0.751, recall 0.796 → F1 0.773
fine: precision 0.893, recall 0.866 → F1 0.880
macro-F1 = (0.773 + 0.880) / 2 = 0.826 ✓ matches data_insights.json

Metric	V1	V2	V3
K-Means silhouette	1.000	0.328	= V2
Baseline macro-F1 (LogReg)	1.000	0.914	—
Urgency test MSE	0.0137	0.0162	0.0160
Urgency test macro-F1	1.000	0.821	0.825
Critical − low separation	—	0.022	0.298

MSE measures how close the 0 to 1 score is. Macro-F1 measures the yes/no decision after thresholding at 0.5.
V2 and V3 are almost tied on these synthetic test rows. The difference appears only on text neither model trained on.

Reading V1F1 = 1.0 and silhouette = 1.0 came from 9 unique sentences. The numbers fell to realistic levels only when the data became realistic.

Theory sheet →Precision, recall, F1 & macro-F1 MSE loss & regression Silhouette score Class imbalance & stratified sampling Data leakage & the train / validation / test split

Stage 4 of 5 · DATA → DB → TRAIN → EVAL → APP

Does the fix hold on real text? Separation, Mann–Whitney and Spearman

I measured it on 8 hand-written sentences, then on 150 real Yelp reviews and 150 real tweets per class.

scripts/stress_test_compare.pyscripts/stress_test_compare.py · L46–49
46def sep(key):
47    crit = [r[key] for r in results if r["expected"] == "CRITICAL"]
48    low = [r[key] for r in results if r["expected"] == "LOW"]
49    return round(sum(crit) / len(crit) - sum(low) / len(low), 4)

scripts/validate_real_data.pyscripts/validate_real_data.py · L61–66
61inverted_rating = (4 - all_sample["label"]).tolist()
62v2_corr, v2_p = spearmanr(inverted_rating, v2_all)
63v3_corr, v3_p = spearmanr(inverted_rating, v3_all)
⋯
65v2_u, v2_u_p = mannwhitneyu(v2_crit, v2_low, alternative="greater")
66v3_u, v3_u_p = mannwhitneyu(v3_crit, v3_low, alternative="greater")

Real-data test	V2	V3
Yelp · Mann–Whitney p	0.0048	< 0.0001
Tweets · Mann–Whitney p	0.39	< 0.0001
Yelp · Spearman ρ vs stars	0.07	0.68

What it showsV3 ranks real reviews far better: ρ rises from 0.07 to 0.68. On tweets V2 is indistinguishable from noise (p = 0.39). The ~13× on 8 sentences shrinks to ~5× on Yelp.

What it does not show

Ranking, not tiers: V3's average score is 0.55 for 1-star and 0.42 for 5-star reviews. Both sit in the mid range.
Star rating and sentiment are proxies for urgency, and only the first 64 tokens of each text are read.
Only 8 stress-test sentences: 3 critical, 2 low.

nested function (closure)list comprehension with filter

Theory sheet →Critical-minus-low separation Mann–Whitney U test Spearman rank correlation Hypothesis test & p-value Transfer learning, fine-tuning & freezing

Stage 5 of 5 · DATA → DB → TRAIN → EVAL → APP

GenAI copilot: an LLM with guardrails and a fallback

Claude drafts the resolution when a key exists. Otherwise a deterministic local backend returns the same JSON shape, so the demo never breaks.

src/genai_copilot.pysrc/genai_copilot.py · L12–36
12SYSTEM_PROMPT = """You are OmniFeedback AI's Resolution Copilot, an enterprise customer support \
13triage assistant. Given one piece of customer feedback and its predicted urgency score (0.0=calm, \
141.0=critical), think step by step about the root cause, then respond with ONLY a single valid JSON \
15object (no markdown fences, no commentary) matching exactly this schema:
16{"thought_process": "...", "root_cause": "...", "risk_level": "CRITICAL|HIGH|MEDIUM|LOW", \
17"recommended_action": "...", "customer_reply_draft": "..."}
18Keep the customer_reply_draft empathetic, concise (2-4 sentences), and compliant with a professional \
19support tone. Never invent account-specific details you were not given."""
⋯
21FEW_SHOT_EXAMPLES = [
22    {
23        "feedback": "App keeps crashing during checkout payment step!",
24        "urgency_score": 0.91,
25        "response": {
26            "thought_process": "Crash occurs specifically at payment step, blocking revenue-generating "
27                                "transactions; high urgency score confirms broad user impact.",
28            "root_cause": "Payment flow instability, likely a recent regression in the checkout module.",
29            "risk_level": "CRITICAL",
30            "recommended_action": "Escalate to on-call payments engineering immediately; check recent "
31                                   "checkout deploys for regressions; consider rollback.",
32            "customer_reply_draft": "We're sorry for the trouble at checkout. Our engineering team has "
33                                     "been alerted and is investigating urgently. We'll follow up as soon "
34                                     "as this is resolved.",
35        },
36    },

src/genai_copilot.pysrc/genai_copilot.py · L159–172
159api_key = _get_anthropic_key()
160backend_used = "local_rules"
161if api_key:
162    try:
163        result = _claude_backend(feedback_text, urgency_score, api_key)
164        backend_used = "claude"
165        result["_backend"] = backend_used
166        return result
167    except Exception:
168        pass  # fall through to local backend
169 
170result = _local_backend(feedback_text, urgency_score)
171result["_backend"] = backend_used
172return result

src/genai_copilot.pysrc/genai_copilot.py · L138–149
138def _parse_json_response(raw: str) -> dict:
  """Extract and validate the first JSON object in a response, tolerating markdown fences."""
  text = raw.strip()
  text = re.sub(r"^```(?:json)?|```$", "", text, flags=re.MULTILINE).strip()
  match = re.search(r"\{.*\}", text, re.DOTALL)
  if not match:
      raise ValueError("No JSON object found in LLM response")
  data = json.loads(match.group(0))
  required_keys = {"thought_process", "root_cause", "risk_level", "recommended_action", "customer_reply_draft"}
  if not required_keys.issubset(data.keys()):
      raise ValueError(f"LLM response missing keys: {required_keys - data.keys()}")
  return data

src/transformer_nlp.pysrc/transformer_nlp.py · L21–47
21@lru_cache(maxsize=1)
22def get_ner_pipeline():
  from transformers import pipeline
  # device=-1 pins to CPU explicitly (Streamlit Community Cloud has no GPU; this also
  # silences the pipeline's "accelerator available but no device passed" warning on
  # machines that do have one, e.g. Apple Silicon MPS during local dev).
  return pipeline("ner", model=NER_MODEL_NAME, aggregation_strategy="simple", device=-1)
⋯
39def extract_entities(text: str) -> list:
  if not text or not text.strip():
      return []
  ner = get_ner_pipeline()
  results = ner(text)
  return [
      {"entity_group": r["entity_group"], "word": r["word"], "score": float(r["score"])}
      for r in results
  ]

Score
urgency score from the model (slide 8 dispatcher)
Prompt
system prompt + 2 few-shot examples; thought_process first (chain of thought)
Claude
model claude-sonnet-4-5 via the Anthropic SDK
Validate
strip fences, parse, check 5 required keys
Fallback
any failure → _local_backend, tagged _backend

Real output of the local backend (score 0.90)

generated at build time by calling _local_backend
{
  "risk_level": "CRITICAL",
  "root_cause": "Billing/payment processing discrepancy.",
  "recommended_action": "Escalate to the relevant on-call team within the SLA window and monitor for repeat reports."
}

LLM limits I designed aroundIt can hallucinate, return malformed JSON, and be steered by text pasted into the prompt. Hence a strict schema, a validating parser, a human reading the output, and a fallback.

try / except fallbacklazy import: anthropic@lru_cache singleton

Theory sheet →LLM Claude Prompt engineering: system prompt, few-shot & chain-of-thought Structured JSON output & validation Pluggable backend & graceful degradation Named Entity Recognition Summarisation: extractive vs abstractive Hugging Face pipelines & the lru_cache singleton

Stage 5 of 5 · DATA → DB → TRAIN → EVAL → APP

The app, the tests, and what I would improve next

A seven-page Streamlit dashboard on top of everything above, with 31 tests and a Docker image.

src/app_state.pysrc/app_state.py · L11–27
11def select_variant() -> dict:
  if "variant_id" not in st.session_state:
      st.session_state["variant_id"] = DEFAULT_VARIANT

  with st.sidebar:
      st.markdown("### 🔀 Dataset / Model Version")
      variant_id = st.radio(
          "Compare naive vs. hardened training",
          options=list(VARIANTS.keys()),
          format_func=lambda v: VARIANTS[v]["short_label"],
          index=list(VARIANTS.keys()).index(st.session_state["variant_id"]),
          key="variant_id",
      )
      st.caption(VARIANTS[variant_id]["description"])
      st.divider()

  return VARIANTS[variant_id]

app.pyapp.py · L17–23
17@st.cache_resource
18def get_db(db_path):
19    return WarehouseManager(db_path=db_path)
⋯
22def load_feedback(db_path):
23    return get_db(db_path).fetch_all_feedback()

tests/test_database.pytests/test_database.py · L13–53
13@pytest.fixture
14def db():
15    with tempfile.TemporaryDirectory() as tmp:
16        yield WarehouseManager(db_path=Path(tmp) / "test.db")
⋯
47def test_rollback_on_bad_urgency_score(db):
48    df = _sample_df(2)
49    df.loc[1, "urgency_score"] = 5.0  # violates CHECK constraint (0.0-1.0)
50    with pytest.raises(Exception):
51        db.insert_feedback_batch(df)
52    result = db.fetch_all_feedback()
53    assert len(result) == 0  # entire batch rolled back, nothing partially committed

Live app ↗ GitHub ↗Theory sheet →

dashboard pages

pytest tests

model variants, toggled live

What I took from it

A perfect metric deserves suspicion; check the data before the model.
Fix a generalisation problem with the right tool: here, pretrained knowledge.
Build in a fallback so a demo never fails, but make the fallback visible.

What I would improve next

One shared risk-threshold constant: the code currently uses five different cut-offs.
Log the exception hidden by the Claude fallback, and keep the model name in one constant.
Run data_insights.py on held-out rows only (it is in-sample today).
Fill predicted_cluster in the warehouse, or remove the unused column.

st.session_state@st.cache_resourcepytest fixturesDockerfile

Theory sheet →Streamlit pytest: fixtures, monkeypatch & mocks Docker Registry & dispatcher pattern Pluggable backend & graceful degradation

Hi, I'm Pranav Chaturvedi.

The problem, and the solution I applied

The problem

What I built

Same pipeline, three versions: the fix made visible

The pipeline, and how each slide reads

Generate & clean

Store safely

Learn

Judge honestly

Use it

How to read a slide

Reproduce it (from the README)

Generating data: the V1 trap and the V2 fix

What the code does

Cleaning: one shared function, called everywhere

A real warehouse row

Replaying clean_text on a messy input (real class, real output)

● SHARED FUNCTION called from 7 places (outside tests)

Star schema: one fact table, small dimension tables

Dim_User (user_id)

Dim_Channel (id, name)

Dim_Product (id, aspect)

Fact_Feedback (feedback_id, user_id, channel_id, aspect_id, texts, score, …)

ACID: a batch either lands completely or not at all

Fact_Feedback (what the dashboard can see)

Fact_Feedback (what the dashboard can see)

Inheritance and shared functions: how the code is organised

◆ INHERITANCE four classes

● SHARED FUNCTION global / shared functions

Classical ML: TF-IDF, K-Means and the Kneedle elbow

Kneedle on the project's own test case

Deep learning: the BiLSTM, written from scratch in PyTorch

How it flows

V3: transfer learning with a pretrained DistilBERT

Evaluation: macro-F1, MSE and silhouette, read honestly

How macro-F1 is computed (V2, all 4,000 rows, in-sample)

Does the fix hold on real text? Separation, Mann–Whitney and Spearman

GenAI copilot: an LLM with guardrails and a fallback

Real output of the local backend (score 0.90)

The app, the tests, and what I would improve next

What I took from it

What I would improve next

Replaying `clean_text` on a messy input (real class, real output)