Image-occlusion diagram extraction fails with Cannot find module 'pdfjs-dist/package.json' on the server

Title: Image-occlusion diagram extraction fails with Cannot find module 'pdfjs-dist/package.json' on the server

Product / Feature: Studio — Image Occlusion flashcard creation (diagram extraction from PDF sources)

Severity: High (feature is fully blocked — no diagram cards can be generated) Platform: Web (serverless / edge runtime)

Summary

Attempting to create an image-occlusion flashcard set from a PDF notebook source (e.g., a 65-page lecture PDF) fails every time at the diagram-detection step. The extraction never runs; the request dies with a server-side Node module-resolution error before any page is parsed.

Environment / Steps to reproduce

Select a PDF source in a notebook (confirmed on a 65-page lecture deck).
Request image-occlusion flashcards from the labeled diagrams (hide-all mode, all diagrams).
Observe: the diagram-extraction service returns an error on every attempt, across multiple pages and multiple retries.

Expected behavior

Diagram detection runs on the requested pages and returns labeled figures so an image-occlusion flashcard set can be generated.

Actual behavior

Every extraction attempt fails with the following error:

Cannot find module 'pdfjs-dist/package.json'
Require stack: - /var/task/apps/web/package.json

The failure occurs in the serverless function (/var/task/apps/web), before any PDF page parsing or diagram detection happens.

Root-cause diagnosis

The error indicates that pdf.js (pdfjs-dist) is being incorrectly resolved and bundled on the server side. In Next.js deployments, pdfjs-dist is not being treated as an external server dependency; instead, Next.js attempts to bundle/optimize it for the serverless runtime, and the bundler cannot resolve pdfjs-dist/package.json from the app package at /var/task/apps/web. Because image-occlusion generation depends on pdf.js to render PDF pages, the entire feature is blocked.

Suggested fix (Option 1 — recommended): treat pdf.js as an external server dependency

Modify next.config.js (or next.config.mjs) to exclude pdfjs-dist from server bundling:

/** @type {import('next').NextConfig} */
const nextConfig = {
  // Force pdfjs-dist to be treated as an external module; skip Serverless bundle optimization
  serverExternalPackages: ['pdfjs-dist'],
};

module.exports = nextConfig;

Note: On Next.js versions before 15, the equivalent is experimental.serverComponentsExternalPackages: ['pdfjs-dist'].

Suggested fix (Option 2 — point the pdf.js worker at the bundled .mjs build)

Ensure the pdf.js worker resolves correctly in the bundler context by importing the legacy build and setting the worker source to the ESM bundle:

import * as pdfjsLib from "pdfjs-dist/legacy/build/pdf";
pdfjsLib.GlobalWorkerOptions.workerSrc = new URL(
  "pdfjs-dist/build/pdf.worker.min.mjs",
  import.meta.url,
).toString();

This addresses the same class of module-resolution/bundling failure that breaks pdf.js at runtime.

Impact

Users cannot create any image-occlusion flashcards from PDF sources while this issue persists.
The bug reproduces consistently and blocks a primary flashcard-generation workflow in Studio.

Supporting evidence

Exact error string: Cannot find module 'pdfjs-dist/package.json'
Require stack resolves to: /var/task/apps/web/package.json
Confirmed reproducible on multiple pages and multiple extraction attempts.

A few quick notes:

I can't apply this fix myself — it's a platform source-code/redeploy change that only the Memo engineering team can make (I have no tool to edit the app's build config, and doing so would require a redeployment).
Once the fix lands, the diagram-detection service should recover, and I can immediately generate your all-diagram, hide-all-show-one occlusion set from Lecture 1 in a single pass.