MUI Docs Infra

Warning

This is an internal project, and is not intended for public use. No support or stability guarantees are provided.

Parse Source

The parseSource utility parses source code into HAST (Hypertext Abstract Syntax Tree) nodes with syntax highlighting using Starry Night. It converts code into highlighted HTML structures for display in documentation and demos.

Basic Usage

 
import { createParseSource } from '@mui/internal-docs-infra/pipeline/parseSource';

// Initialize the parser (do this once, typically at app startup)
const parseSource = await createParseSource();

// Parse and highlight JavaScript code
const highlighted = parseSource('const x = 42;', 'example.js');

// Use the HAST tree for rendering (e.g., with hastToReact)

The parser automatically:

  • Detects the language from the file extension
  • Applies syntax highlighting
  • Adds line gutter elements
  • Returns a HAST tree ready for rendering

Common Patterns

Highlighting Different Languages

 
import { createParseSource } from '@mui/internal-docs-infra/pipeline/parseSource';

const parseSource = await createParseSource();

// JavaScript
const jsCode = parseSource('const x = 42;', 'example.js');

// TypeScript
const tsCode = parseSource('interface User { name: string; }', 'types.ts');

// CSS
const cssCode = parseSource('.button { color: blue; }', 'styles.css');

// HTML
const htmlCode = parseSource('<div>Hello</div>', 'index.html');

Processing Multiple Files

 
import { createParseSource } from '@mui/internal-docs-infra/pipeline/parseSource';

const parseSource = await createParseSource();

const files = [
  { name: 'App.tsx', content: 'export default function App() {}' },
  { name: 'styles.css', content: '.app { margin: 0; }' },
  { name: 'utils.ts', content: 'export const helper = () => {};' },
];

const highlighted = files.map((file) => ({
  name: file.name,
  hast: parseSource(file.content, file.name),
}));

Unsupported File Types

For unsupported file extensions, the parser returns plain text:

const parseSource = await createParseSource();

// Unsupported extension - returns plain text node
const result = parseSource('Some content', 'file.xyz');
// {
//   type: 'root',
//   children: [{ type: 'text', value: 'Some content' }]
// }

// File without extension - also returns plain text
const readme = parseSource('# README', 'README');

Advanced Usage

Global Instance Pattern

After initialization, parseSource can be used directly from anywhere:

import { createParseSource, parseSource } from '@mui/internal-docs-infra/pipeline/parseSource';

// Initialize once (e.g., in app startup)
await createParseSource();

// Use the global instance anywhere in your code
function highlightCode(code: string, fileName: string) {
  return parseSource(code, fileName); // Works without re-initialization
}

Integration with Code Highlighter

 
import { createParseSource } from '@mui/internal-docs-infra/pipeline/parseSource';
import { hastToReact } from '@mui/internal-docs-infra/hastUtils';

const parseSource = await createParseSource();

function HighlightedCode({ code, fileName }: { code: string; fileName: string }) {
  const hast = parseSource(code, fileName);
  const jsx = hastToReact(hast);

  return <pre>{jsx}</pre>;
}

How It Works
  1. Initialization: createParseSource() creates a Starry Night instance with grammar definitions
  2. Language Detection: File extension is extracted and mapped to a grammar
  3. Syntax Highlighting: Starry Night processes the source code and generates HAST nodes with pl-* classes
  4. Syntax Extensions: Adds fine-grained di-* classes (numbers, booleans, nullish, attributes) without removing existing classes
  5. Line Gutters: starryNightGutter() adds line number elements to the tree
  6. Fallback: Unsupported file types return a simple text node

The Starry Night instance is cached globally using __docs_infra_starry_night_instance__ to avoid re-initialization.

Syntax Token Classes

Starry Night maps TextMate scopes to a small set of pl-* CSS classes (the GitHub Prettylights theme). Many token types share the same class — pl-c1 covers numbers, booleans, null, component names, and more.

During parsing, parseSource extends the highlighted tree with additive di-* classes that distinguish these cases, giving CSS full control over each token type:

ClassMeaningExample tokens
di-numNumeric constant42, 3.14, 0xFF, div { width: 100px }
di-boolBooleantrue, false
di-nNullish valuenull, undefined, "", ''
di-thisSelf-reference keywordthis, super
di-btBuilt-in type keywordstring, number, boolean, void, never
di-jsxJSX component elementButton in <Button />, </Button>
di-akAttribute keyclassName in <div className="x">
di-aeAttribute equals= in <div className="x">
di-avAttribute value"x" in <div className="x">
di-daCSS data attribute selectordata-active in [data-active]
di-cpCSS property namecolor in div { color: red }
di-cvCSS property valueflex in div { display: flex }
di-htHTML tag wrapper (inline only)<div>, </span> (wraps brackets + tag name)
di-jtJSX component tag wrapper (inline only)<Box>, </Stack> (wraps brackets + tag name)

di-this applies to JS/TS family grammars (.js, .ts, .tsx, .jsx, .mdx). di-bt applies to TS family grammars only (.ts, .tsx, .jsx, .mdx) — plain JS is excluded because string, number, etc. are valid variable names. di-jsx only applies to JSX grammars (.tsx, .jsx, .mdx). di-ak, di-ae, di-av only apply to HTML/JSX/MDX grammars. di-da, di-cp, and di-cv only apply to CSS. di-ht and di-jt are added by enhanceCodeInline (not parseSource) as wrapper spans around tag brackets and the inner pl-ent or pl-c1 span.

Existing pl-* classes are never removed — <span class="pl-c1">42</span> becomes <span class="pl-c1 di-num">42</span>.

Supported Languages

The parser supports languages based on file extensions:

LanguageExtensions
JavaScript.js, .mjs, .cjs, .jsx
TypeScript.ts, .tsx
CSS.css
HTML.html, .htm
JSON.json
And moreVia Starry Night grammars

See grammars.ts for the complete list of supported languages.

Error Handling

Uninitialized Parser

 
import { parseSource } from '@mui/internal-docs-infra/pipeline/parseSource';

// ❌ This will throw an error
parseSource('code', 'file.js');
// Error: Starry Night not initialized. Use createParseSource to create an initialized parseSource function.

// ✓ Initialize first
import { createParseSource } from '@mui/internal-docs-infra/pipeline/parseSource';
await createParseSource();
parseSource('code', 'file.js'); // Now works

Empty Content

 
const parseSource = await createParseSource();

// Handles empty content gracefully
const result = parseSource('', 'empty.js');
// Returns valid HAST tree with empty content

When to Use
  • Syntax highlighting - When you need to highlight code for display
  • Code documentation - Preparing code examples for docs
  • Build-time processing - Pre-highlighting code during build
  • Custom code viewers - Building code display components

When NOT to use:

  • Simple text display - Use plain <pre> tags if highlighting isn't needed

Types

createParseSource

Initializes Starry Night and returns a configured parseSource function. This only needs to be called once per application. The Starry Night instance is stored globally for reuse across calls.

Return Type
Promise<ParseSource>

A Promise that resolves to the initialized parseSource function

parseSource

Parses source code into a HAST tree with syntax highlighting.

ParameterTypeDescription
source
string
fileName
string
language
string | undefined
Return Type
HastRoot

HAST Root node containing highlighted code structure with line gutters

getGrammarFromLanguage

Gets the grammar scope from a language name.

ParameterTypeDescription
language
string

The language name (e.g., ‘tsx’, ‘css’, ‘typescript’)

Return Type
string | undefined

The grammar scope or undefined if not recognized

Additional Types
extensionMap
type extensionMap = Record<string, string>
languageToGrammarMap

Maps simplified language names back to grammar scope names. Used when language prop is provided instead of fileName.

type languageToGrammarMap = Record<string, string>