Deno overhaul

[bleonar5]

Refactor codebase for cross-platform compatibility and npm publication

Overview

This pull request represents a major overhaul of our codebase, transitioning from Deno-specific implementations to more cross-platform compatible solutions. The primary goal is to prepare our project for a dnt (Deno to Node.js) build and eventual publication to npm.

Key Changes

1. Dependencies: Resolved jsonld-related issues using import_map.json.
2. Assertions: Updated assertion utilities to be Node.js compatible.
3. CLI: Replaced Deno-specific CLI utilities with npm:commander for improved argument handling.
4. Logging: Implemented a custom Winston-based logger to replace Deno's logger.
5. File System Operations: Transitioned to node:fs and node:path for better Node.js compatibility.
6. Schema Loading: Refactored loadSchema module for environment-agnostic path resolution and enhanced error handling.
7. CSV Parsing: Replaced Deno's std/csv with npm:csv-parse/sync for improved robustness and cross-platform support.
8. Configuration Management: Added deno.json for centralized Deno configuration and automatic import map usage. Replaced separate importmap.json with direct imports in deno.json.
9. Linting and Testing: Updated linter configuration in deno.json to exclude non-Deno files. Modified tests to use mock responses instead of real fetches for improved reliability and speed.
10. Code Refactoring: Refactored validator for cross-environment compatibility, separating concerns between Deno and npm environments. Removed top-level awaits to comply with dnt build requirements.
11. NPM Build Process: Added build_npm.ts and shims.ts to transform Deno codebase into npm-compatible packages, enabling creation of ESM and CommonJS versions for npm distribution.
12. Bug Fixes: Resolved issues with dataset directory argument handling in validate.ts and standardized debug level casing in options parsing.

Impact

• Improved cross-platform compatibility, enabling the project to run in both Deno and Node.js environments.
• Enhanced error handling and logging across multiple modules.
• Increased test coverage and improved test reliability.
• Prepared the groundwork for npm package publication, including support for both ESM and CommonJS.
• Streamlined configuration management and import handling.
• Improved CLI functionality and argument processing.

Testing

All existing tests have been updated to work with the new implementations. Additional tests have been added where necessary to ensure functionality and improve coverage. Test reliability has been enhanced by using mock responses where appropriate.

[bleonar5]

@becky-gilbert You mentioned that you would be pulling this down and testing it, so here is some context for that:

(switch to node v20)

You can run the validator normally by installing deno and running: deno run -A stc/psychds-validator.ts <dataset>, and you can add warnings with the -w flag.

To run the dnt build script: deno run -A build_npm.ts 1.0.0

If this runs successfully, try installing the npm CLI tool locally:

cd npm
npm pack 
npm install -g psychds-validator-1.0.0.tgz              
validate ../test_data/valid_datasets/bfi-dataset

If you want, you can also test the esm and cjs imports:

(from the root of psychds-validator)

mkdir test
cd test
npm init -y
npm install ../npm/psychds-validator-1.0.0.tgz

add these test files:

(test-cjs.cjs)

// test-cjs.cjs
const { validate } = require('psychds-validator');

async function runTest() {
    try {
        const result = await validate('../test_data/valid_datasets/bfi-dataset');
        console.log('CJS Test Result:', result);
    } catch (error) {
        console.error('CJS Test Error:', error);
    }
}

runTest();

and (test-esm.mjs)

// test-esm.mjs
import { validate } from 'psychds-validator';

async function runTest() {
    try {
        const result = await validate('../test_data/valid_datasets/bfi-dataset');
        console.log('ESM Test Result:', result);
    } catch (error) {
        console.error('ESM Test Error:', error);
    }
}

runTest();

and then run them:

node test-esm.mjs
node test-cjs.cjs