Context Building

Context building is a core concept in DeepLint that enables intelligent code analysis by providing the LLM with a comprehensive understanding of your codebase.

What is Context Building?

Context building is the process of gathering relevant information about your codebase to provide the LLM with the context it needs to perform semantic analysis. This includes:

• Changed files and their content

• Related files affected by the changes

• Dependencies and relationships between components

• Code structure and organization

• Repository information

Why Context Matters

Traditional linters analyze files in isolation, which limits their ability to detect issues that span multiple files or depend on the relationships between components. DeepLint's context building approach addresses this limitation by:

1. Understanding Cross-File Dependencies: Identifying how changes in one file affect other parts of the codebase

2. Capturing Semantic Relationships: Recognizing the meaning and purpose of code beyond its syntax

3. Providing Holistic View: Giving the LLM a comprehensive view of the codebase structure

Context Building Process

The context building process involves several steps:

1. Git Integration

DeepLint integrates with Git to identify changes in your codebase. It can analyze both staged and unstaged changes, depending on your configuration.

2. Repository Scanning

DeepLint scans your repository to understand its structure, collecting information about:

• Directory hierarchy

• File types and locations

• Project organization

• File metadata (size, last modified date, etc.)

3. Dependency Analysis

DeepLint analyzes dependencies between files to identify relationships, such as:

• Import/export relationships

• Function calls between files

• Type references

• Component usage

4. Code Structure Analysis

DeepLint extracts information about the structure of your code, including:

• Functions and their signatures

• Classes and their methods

• Interfaces and types

• Exports and imports

5. Context Assembly

DeepLint assembles all this information into a structured context that includes:

• Repository information (name, root, structure)

• Changes (files, diffs, summary)

• Related files (dependencies, content, structure)

• Metadata (token counts, generation time, context type)

6. Token Management

LLMs have token limits, so DeepLint manages the context size by:

• Counting tokens using the tiktoken library

• Tracking used and available tokens

• Truncating file content when necessary

• Prioritizing important parts of files (imports, exports, function signatures)

Context Content

The context built by DeepLint includes:

• Changed files with their content

• Dependencies of changed files (when includeDependencies is true, which is the default)

• Code structure information (when includeStructure is true)

• Basic repository structure

For JavaScript and TypeScript projects, DeepLint automatically analyzes dependencies between files. For other project types, dependency analysis is skipped, but the context still includes changed files and repository structure.

Customizing Context Building

You can customize the context building process through configuration:

contextBuilder: {
  // Maximum number of tokens to use for the context
  maxTokens: 8000,

  // Maximum number of tokens to use per file
  tokensPerFile: 1000,

  // Maximum file size in KB to include in the context
  maxFileSize: 500,

  // Whether to include dependencies in the context (default: true)
  includeDependencies: true,

  // Whether to include code structure in the context
  includeStructure: true,
}

Debugging Context Building

You can debug the context building process using the --debug flag:

deeplint --debug

This will show detailed information about the context building process, including:

• Files being scanned

• Dependencies being analyzed

• Token usage

• Context assembly

You can also save the context to a file for inspection using the --dump option:

deeplint --dump context.json

This is useful for understanding what information is being sent to the LLM and for troubleshooting issues.

For more detailed information about the implementation of the Context Builder, see the Context Builder Overview page in the developer documentation.