Refactor metadata storage to use hashes instead of dates

Refactor Metadata Storage to Use Hashes Instead of Dates

Current Behavior:

At present, Docuforge stores metadata about each documented file in a metadata.json file, which includes two properties:

lastDocumented: A timestamp representing when the file was last documented.
lastModified: A timestamp representing the last time the file was modified.

Example of the current format:

{
  "src/infra/jobs/bull/jobs/backup.ts": {
    "lastDocumented": 1727136400462,
    "lastModified": 1727135896603.5308
  }
}

Problem:

Using timestamps (lastModified and lastDocumented) leads to issues where trivial actions, such as saving a file in an editor (e.g., pressing Command + S in VSCode without any actual changes), update the lastModified timestamp. This causes unnecessary regeneration of documentation, which can be inefficient for larger projects.

Proposed Solution:

To avoid unnecessary documentation generation and make the metadata system more accurate, we propose replacing both lastModified and lastDocumented timestamps with hashes of the file content. By comparing these hashes, we can determine if the content has truly changed, reducing false positives for documentation regeneration.

New Metadata Format:

The new metadata.json structure will store two hashes:

documentedHash: The hash of the file content at the time the documentation was last generated.
currentHash: The current hash of the file content, used to determine if it has changed since the last documentation.

Example of the updated format:

{
  "src/infra/jobs/bull/jobs/backup.ts": {
    "documentedHash": "5d41402abc4b2a76b9719d911017c592",
    "currentHash": "7d793037a0760186574b0282f2f435e7"
  }
}

Benefits:

Accuracy: This change ensures that documentation is only regenerated when there are actual content changes, not when timestamps are modified by trivial actions like saving the file.
Efficiency: The system becomes more efficient by avoiding unnecessary processing, especially in large projects with many files.
Consistency: Using hashes for both documentedHash and currentHash ensures a consistent and content-based approach to detecting changes.
Scalability: As projects grow, this method scales better than relying on timestamps, reducing the computational overhead for large-scale documentation tasks.

Implementation Steps:

Replace the use of lastModified with currentHash in the metadata.json file.
Replace the use of lastDocumented with documentedHash in the metadata.json file.
Implement a hash generation function (e.g., using SHA-256 or MD5) to compute the content-based hashes for each file.
Modify the comparison logic to use the hashes (documentedHash vs currentHash) instead of timestamps to decide when to regenerate documentation.
Ensure backward compatibility for older metadata formats, if necessary.

This change will enhance the precision and efficiency of Docuforge's documentation generation process, especially for users working with large or frequently edited codebases.

douglasrubims / docuforge