search

dprint / dprint-plugin-json

JSON code formatting plugin for dprint.
https://dprint.dev/plugins/json
MIT License
18 stars 5 forks source link

Can't formatting over 4 files with `.sh` script #25

Open doox911-opensource opened 1 year ago

doox911-opensource commented 1 year ago

OS

Windows 11 Pro. WSL 2.

11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz 2.80 GHz

RAM 16gb

Dprint version

0.41.0

JSON plugin

https://plugins.dprint.dev/json-0.17.4.wasm

dprint.json

{
  "plugins": [
    "https://plugins.dprint.dev/json-0.17.4.wasm"
  ]
}

My script

# /bin/sh

echo "dprint:"

START=$(date +%s%N)

npx dprint fmt --incremental=false

END=$(date +%s%N)

milliseconds=$((($END - $START)/1000000))

seconds=$(($milliseconds/1000))

minutes=$(($seconds/60))

echo "$milliseconds"
echo "$seconds"
echo "$minutes"

Log

WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_1.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_2.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_3.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_4.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_5.json
WARNING: Formatting is slow for [my_path]/jsonformat/benches/large-file_6.json
....

I created 100 identical files from https://github.com/Nilstrieb/jsonformat/blob/master/benches/large-file.json.

When I format 4 files everything is fine! But when I try to format more, then I see a WARNING and not one file is formatted.

dsherret commented 1 year ago

Can you run with dprint fmt --verbose? For me it formats, but it is very slow for such a large file.

doox911-opensource commented 1 year ago

Can you run with dprint fmt --verbose? For me it formats, but it is very slow for such a large file.

dprint:
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/locks
[VERBOSE] Checking path exists: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64-3707102107370057232
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Globbing: GlobPatterns { includes: Some([GlobPattern { relative_pattern: "**/*.{json,jsonc}", base_dir: CanonicalizedPathBuf { path: "[my_private_path]/jsonformat/benches" } }]), excludes: [GlobPattern { relative_pattern: "!**/node_modules", base_dir: CanonicalizedPathBuf { path: "[my_private_path]/jsonformat/benches" } }] }
[VERBOSE] File(s) matched: GlobOutput { file_paths: ["[my_private_path]/jsonformat/benches/dprint.json", "[my_private_path]/jsonformat/benches/large-file_1.json", "[my_private_path]/jsonformat/benches/large-file_10.json", "[my_private_path]/jsonformat/benches/large-file_100.json", "[my_private_path]/jsonformat/benches/large-file_11.json", "[my_private_path]/jsonformat/benches/large-file_12.json", "[my_private_path]/jsonformat/benches/large-file_13.json", "[my_private_path]/jsonformat/benches/large-file_14.json", "[my_private_path]/jsonformat/benches/large-file_15.json", "[my_private_path]/jsonformat/benches/large-file_16.json", "[my_private_path]/jsonformat/benches/large-file_17.json", "[my_private_path]/jsonformat/benches/large-file_18.json", "[my_private_path]/jsonformat/benches/large-file_19.json", "[my_private_path]/jsonformat/benches/large-file_2.json", "[my_private_path]/jsonformat/benches/large-file_20.json", "[my_private_path]/jsonformat/benches/large-file_21.json", "[my_private_path]/jsonformat/benches/large-file_22.json", "[my_private_path]/jsonformat/benches/large-file_23.json", "[my_private_path]/jsonformat/benches/large-file_24.json", "[my_private_path]/jsonformat/benches/large-file_25.json", "[my_private_path]/jsonformat/benches/large-file_26.json", "[my_private_path]/jsonformat/benches/large-file_27.json", "[my_private_path]/jsonformat/benches/large-file_28.json", "[my_private_path]/jsonformat/benches/large-file_29.json", "[my_private_path]/jsonformat/benches/large-file_3.json", "[my_private_path]/jsonformat/benches/large-file_30.json", "[my_private_path]/jsonformat/benches/large-file_31.json", "[my_private_path]/jsonformat/benches/large-file_32.json", "[my_private_path]/jsonformat/benches/large-file_33.json", "[my_private_path]/jsonformat/benches/large-file_34.json", "[my_private_path]/jsonformat/benches/large-file_35.json", "[my_private_path]/jsonformat/benches/large-file_36.json", "[my_private_path]/jsonformat/benches/large-file_37.json", "[my_private_path]/jsonformat/benches/large-file_38.json", "[my_private_path]/jsonformat/benches/large-file_39.json", "[my_private_path]/jsonformat/benches/large-file_4.json", "[my_private_path]/jsonformat/benches/large-file_40.json", "[my_private_path]/jsonformat/benches/large-file_41.json", "[my_private_path]/jsonformat/benches/large-file_42.json", "[my_private_path]/jsonformat/benches/large-file_43.json", "[my_private_path]/jsonformat/benches/large-file_44.json", "[my_private_path]/jsonformat/benches/large-file_45.json", "[my_private_path]/jsonformat/benches/large-file_46.json", "[my_private_path]/jsonformat/benches/large-file_47.json", "[my_private_path]/jsonformat/benches/large-file_48.json", "[my_private_path]/jsonformat/benches/large-file_49.json", "[my_private_path]/jsonformat/benches/large-file_5.json", "[my_private_path]/jsonformat/benches/large-file_50.json", "[my_private_path]/jsonformat/benches/large-file_51.json", "[my_private_path]/jsonformat/benches/large-file_52.json", "[my_private_path]/jsonformat/benches/large-file_53.json", "[my_private_path]/jsonformat/benches/large-file_54.json", "[my_private_path]/jsonformat/benches/large-file_55.json", "[my_private_path]/jsonformat/benches/large-file_56.json", "[my_private_path]/jsonformat/benches/large-file_57.json", "[my_private_path]/jsonformat/benches/large-file_58.json", "[my_private_path]/jsonformat/benches/large-file_59.json", "[my_private_path]/jsonformat/benches/large-file_6.json", "[my_private_path]/jsonformat/benches/large-file_60.json", "[my_private_path]/jsonformat/benches/large-file_61.json", "[my_private_path]/jsonformat/benches/large-file_62.json", "[my_private_path]/jsonformat/benches/large-file_63.json", "[my_private_path]/jsonformat/benches/large-file_64.json", "[my_private_path]/jsonformat/benches/large-file_65.json", "[my_private_path]/jsonformat/benches/large-file_66.json", "[my_private_path]/jsonformat/benches/large-file_67.json", "[my_private_path]/jsonformat/benches/large-file_68.json", "[my_private_path]/jsonformat/benches/large-file_69.json", "[my_private_path]/jsonformat/benches/large-file_7.json", "[my_private_path]/jsonformat/benches/large-file_70.json", "[my_private_path]/jsonformat/benches/large-file_71.json", "[my_private_path]/jsonformat/benches/large-file_72.json", "[my_private_path]/jsonformat/benches/large-file_73.json", "[my_private_path]/jsonformat/benches/large-file_74.json", "[my_private_path]/jsonformat/benches/large-file_75.json", "[my_private_path]/jsonformat/benches/large-file_76.json", "[my_private_path]/jsonformat/benches/large-file_77.json", "[my_private_path]/jsonformat/benches/large-file_78.json", "[my_private_path]/jsonformat/benches/large-file_79.json", "[my_private_path]/jsonformat/benches/large-file_8.json", "[my_private_path]/jsonformat/benches/large-file_80.json", "[my_private_path]/jsonformat/benches/large-file_81.json", "[my_private_path]/jsonformat/benches/large-file_82.json", "[my_private_path]/jsonformat/benches/large-file_83.json", "[my_private_path]/jsonformat/benches/large-file_84.json", "[my_private_path]/jsonformat/benches/large-file_85.json", "[my_private_path]/jsonformat/benches/large-file_86.json", "[my_private_path]/jsonformat/benches/large-file_87.json", "[my_private_path]/jsonformat/benches/large-file_88.json", "[my_private_path]/jsonformat/benches/large-file_89.json", "[my_private_path]/jsonformat/benches/large-file_9.json", "[my_private_path]/jsonformat/benches/large-file_90.json", "[my_private_path]/jsonformat/benches/large-file_91.json", "[my_private_path]/jsonformat/benches/large-file_92.json", "[my_private_path]/jsonformat/benches/large-file_93.json", "[my_private_path]/jsonformat/benches/large-file_94.json", "[my_private_path]/jsonformat/benches/large-file_95.json", "[my_private_path]/jsonformat/benches/large-file_96.json", "[my_private_path]/jsonformat/benches/large-file_97.json", "[my_private_path]/jsonformat/benches/large-file_98.json", "[my_private_path]/jsonformat/benches/large-file_99.json"], config_files: [] }
[VERBOSE] Finished globbing in 11ms
[VERBOSE] Running for config: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Max threads: 8
Thread count: 7
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/dprint.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_1.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_10.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_100.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_11.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_12.json
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_13.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/dprint.json in 0ms
[VERBOSE] Reading file: [my_private_path]/jsonformat/benches/large-file_14.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 35ms
[VERBOSE] Created instance of dprint-plugin-json in 28ms
[VERBOSE] Created instance of dprint-plugin-json in 21ms
[VERBOSE] Created instance of dprint-plugin-json in 15ms
[VERBOSE] Created instance of dprint-plugin-json in 8ms
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] CPU usage: 88%
[VERBOSE] High CPU. Reducing parallelism.
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_12.json in 5386ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_12.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_14.json in 5414ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_14.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_100.json in 5491ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_100.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_13.json in 5542ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_13.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_10.json in 5580ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_10.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_1.json in 5627ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_1.json
[VERBOSE] Formatted file: [my_private_path]/jsonformat/benches/large-file_11.json in 5669ms
[VERBOSE] Ensuring stable format: [my_private_path]/jsonformat/benches/large-file_11.json
[VERBOSE] CPU usage: 88%
[VERBOSE] CPU usage: 16%
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_1.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_10.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_100.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_11.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_12.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_13.json
WARNING: Formatting is slow for [my_private_path]/jsonformat/benches/large-file_14.json
[VERBOSE] CPU usage: 1%
[VERBOSE] Low CPU. Increasing parallelism.
[VERBOSE] CPU usage: 1%
[VERBOSE] CPU usage: 0%

Then the script script freezes and it is clear about the CPU load that there is no work.

dsherret commented 1 year ago

Ok, that looks like a dprint bug. Thanks for the output. I'll look into it.

dsherret commented 1 year ago

@doox911-opensource I haven't been able to reproduce it yet on multiple machines. When you get a chance, would you be able to try setting the CI env var to 1 (ex. CI=1 npx dprint fmt --verbose --incremental=false)? That will disable CPU throttling. I'm curious if it's caused by that.

If it's too much of an ask, I also wonder if this occurs in dprint 0.39.1 and starts occurring in 0.40.0.

doox911-opensource commented 1 year ago

@doox911-opensource I haven't been able to reproduce it yet on multiple machines. When you get a chance, would you be able to try setting the CI env var to 1 (ex. CI=1 npx dprint fmt --verbose --incremental=false)? That will disable CPU throttling. I'm curious if it's caused by that.

If it's too much of an ask, I also wonder if this occurs in dprint 0.39.1 and starts occurring in 0.40.0.

Script:

# /bin/sh

echo "dprint:"

START=$(date +%s%N)

CI=1 npx dprint fmt --incremental=false --verbose

END=$(date +%s%N)

milliseconds=$((($END - $START)/1000000))

seconds=$(($milliseconds/1000))

minutes=$(($seconds/60))

echo "$milliseconds"
echo "$seconds"
echo "$minutes"

or just a command: 

time npx dprint fmt --incremental=false large-file_*.json

or 

npx dprint fmt --incremental=false large-file_*.json

The problem is observed not only when running dprint in the script

Result:

doox911@doox911:/mnt/c/Users/[my_private_path]$ ./dprint.sh
dprint:
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/locks
[VERBOSE] Checking path exists: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64-3707102107370057232
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Globbing: GlobPatterns { includes: Some([GlobPattern { relative_pattern: "**/*.{json,jsonc}", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_private_path]" } }]), excludes: [GlobPattern { relative_pattern: "!**/node_modules", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_private_path]" } }] }
[VERBOSE] File(s) matched: GlobOutput { file_paths: ["/mnt/c/Users/[my_private_path]/.prettierrc.json", "/mnt/c/Users/[my_private_path]/dprint.json", "/mnt/c/Users/[my_private_path]/large-file_1.json", "/mnt/c/Users/[my_private_path]/large-file_10.json", "/mnt/c/Users/[my_private_path]/large-file_100.json", "/mnt/c/Users/[my_private_path]/large-file_11.json", "/mnt/c/Users/[my_private_path]/large-file_12.json", "/mnt/c/Users/[my_private_path]/large-file_13.json", "/mnt/c/Users/[my_private_path]/large-file_14.json", "/mnt/c/Users/[my_private_path]/large-file_15.json", "/mnt/c/Users/[my_private_path]/large-file_16.json", "/mnt/c/Users/[my_private_path]/large-file_17.json", "/mnt/c/Users/[my_private_path]/large-file_18.json", "/mnt/c/Users/[my_private_path]/large-file_19.json", "/mnt/c/Users/[my_private_path]/large-file_2.json", "/mnt/c/Users/[my_private_path]/large-file_20.json", "/mnt/c/Users/[my_private_path]/large-file_21.json", "/mnt/c/Users/[my_private_path]/large-file_22.json", "/mnt/c/Users/[my_private_path]/large-file_23.json", "/mnt/c/Users/[my_private_path]/large-file_24.json", "/mnt/c/Users/[my_private_path]/large-file_25.json", "/mnt/c/Users/[my_private_path]/large-file_26.json", "/mnt/c/Users/[my_private_path]/large-file_27.json", "/mnt/c/Users/[my_private_path]/large-file_28.json", "/mnt/c/Users/[my_private_path]/large-file_29.json", "/mnt/c/Users/[my_private_path]/large-file_3.json", "/mnt/c/Users/[my_private_path]/large-file_30.json", "/mnt/c/Users/[my_private_path]/large-file_31.json", "/mnt/c/Users/[my_private_path]/large-file_32.json", "/mnt/c/Users/[my_private_path]/large-file_33.json", "/mnt/c/Users/[my_private_path]/large-file_34.json", "/mnt/c/Users/[my_private_path]/large-file_35.json", "/mnt/c/Users/[my_private_path]/large-file_36.json", "/mnt/c/Users/[my_private_path]/large-file_37.json", "/mnt/c/Users/[my_private_path]/large-file_38.json", "/mnt/c/Users/[my_private_path]/large-file_39.json", "/mnt/c/Users/[my_private_path]/large-file_4.json", "/mnt/c/Users/[my_private_path]/large-file_40.json", "/mnt/c/Users/[my_private_path]/large-file_41.json", "/mnt/c/Users/[my_private_path]/large-file_42.json", "/mnt/c/Users/[my_private_path]/large-file_43.json", "/mnt/c/Users/[my_private_path]/large-file_44.json", "/mnt/c/Users/[my_private_path]/large-file_45.json", "/mnt/c/Users/[my_private_path]/large-file_46.json", "/mnt/c/Users/[my_private_path]/large-file_47.json", "/mnt/c/Users/[my_private_path]/large-file_48.json", "/mnt/c/Users/[my_private_path]/large-file_49.json", "/mnt/c/Users/[my_private_path]/large-file_5.json", "/mnt/c/Users/[my_private_path]/large-file_50.json", "/mnt/c/Users/[my_private_path]/large-file_51.json", "/mnt/c/Users/[my_private_path]/large-file_52.json", "/mnt/c/Users/[my_private_path]/large-file_53.json", "/mnt/c/Users/[my_private_path]/large-file_54.json", "/mnt/c/Users/[my_private_path]/large-file_55.json", "/mnt/c/Users/[my_private_path]/large-file_56.json", "/mnt/c/Users/[my_private_path]/large-file_57.json", "/mnt/c/Users/[my_private_path]/large-file_58.json", "/mnt/c/Users/[my_private_path]/large-file_59.json", "/mnt/c/Users/[my_private_path]/large-file_6.json", "/mnt/c/Users/[my_private_path]/large-file_60.json", "/mnt/c/Users/[my_private_path]/large-file_61.json", "/mnt/c/Users/[my_private_path]/large-file_62.json", "/mnt/c/Users/[my_private_path]/large-file_63.json", "/mnt/c/Users/[my_private_path]/large-file_64.json", "/mnt/c/Users/[my_private_path]/large-file_65.json", "/mnt/c/Users/[my_private_path]/large-file_66.json", "/mnt/c/Users/[my_private_path]/large-file_67.json", "/mnt/c/Users/[my_private_path]/large-file_68.json", "/mnt/c/Users/[my_private_path]/large-file_69.json", "/mnt/c/Users/[my_private_path]/large-file_7.json", "/mnt/c/Users/[my_private_path]/large-file_70.json", "/mnt/c/Users/[my_private_path]/large-file_71.json", "/mnt/c/Users/[my_private_path]/large-file_72.json", "/mnt/c/Users/[my_private_path]/large-file_73.json", "/mnt/c/Users/[my_private_path]/large-file_74.json", "/mnt/c/Users/[my_private_path]/large-file_75.json", "/mnt/c/Users/[my_private_path]/large-file_76.json", "/mnt/c/Users/[my_private_path]/large-file_77.json", "/mnt/c/Users/[my_private_path]/large-file_78.json", "/mnt/c/Users/[my_private_path]/large-file_79.json", "/mnt/c/Users/[my_private_path]/large-file_8.json", "/mnt/c/Users/[my_private_path]/large-file_80.json", "/mnt/c/Users/[my_private_path]/large-file_81.json", "/mnt/c/Users/[my_private_path]/large-file_82.json", "/mnt/c/Users/[my_private_path]/large-file_83.json", "/mnt/c/Users/[my_private_path]/large-file_84.json", "/mnt/c/Users/[my_private_path]/large-file_85.json", "/mnt/c/Users/[my_private_path]/large-file_86.json", "/mnt/c/Users/[my_private_path]/large-file_87.json", "/mnt/c/Users/[my_private_path]/large-file_88.json", "/mnt/c/Users/[my_private_path]/large-file_89.json", "/mnt/c/Users/[my_private_path]/large-file_9.json", "/mnt/c/Users/[my_private_path]/large-file_90.json", "/mnt/c/Users/[my_private_path]/large-file_91.json", "/mnt/c/Users/[my_private_path]/large-file_92.json", "/mnt/c/Users/[my_private_path]/large-file_93.json", "/mnt/c/Users/[my_private_path]/large-file_94.json", "/mnt/c/Users/[my_private_path]/large-file_95.json", "/mnt/c/Users/[my_private_path]/large-file_96.json", "/mnt/c/Users/[my_private_path]/large-file_97.json", "/mnt/c/Users/[my_private_path]/large-file_98.json", "/mnt/c/Users/[my_private_path]/large-file_99.json", "/mnt/c/Users/[my_private_path]/package.json"], config_files: [] }
[VERBOSE] Finished globbing in 13ms
[VERBOSE] Running for config: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Max threads: 8
Thread count: 7
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/.prettierrc.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_10.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_1.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_100.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_11.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_12.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/dprint.json in 0ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/.prettierrc.json in 0ms
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_13.json
[VERBOSE] Reading file: /mnt/c/Users/[my_private_path]/large-file_14.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 37ms
[VERBOSE] Created instance of dprint-plugin-json in 29ms
[VERBOSE] Created instance of dprint-plugin-json in 23ms
[VERBOSE] Created instance of dprint-plugin-json in 14ms
[VERBOSE] Created instance of dprint-plugin-json in 8ms
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_10.json in 6465ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_10.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_12.json in 6471ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_12.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_100.json in 6546ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_100.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_14.json in 6578ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_14.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_11.json in 6729ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_11.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_1.json in 6841ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_1.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_private_path]/large-file_13.json in 6958ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_private_path]/large-file_13.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_1.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_10.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_100.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_11.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_12.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_13.json
WARNING: Formatting is slow for /mnt/c/Users/[my_private_path]/large-file_14.json
23096
23
0

For dprint 0.39.1:

doox911@doox911:/mnt/c/Users/[my_path]$ ./dprint.sh
dprint:
dprint 0.39.1
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Busting plugins cache due to deserialization error: missing field `fileExtensions` at line 1 column 425
[VERBOSE] Deleting directory: /home/doox911/.cache/dprint/cache/plugins
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/locks
[VERBOSE] Checking path exists: /mnt/c/Users/[my_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/dprint.json
[VERBOSE] Acquiring file lock at /home/doox911/.cache/dprint/cache/locks/.3615629463447869408.lock
[VERBOSE] Acquired file lock at /home/doox911/.cache/dprint/cache/locks/.3615629463447869408.lock
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Busting plugins cache due to deserialization error: missing field `fileExtensions` at line 1 column 425
[VERBOSE] Deleting directory: /home/doox911/.cache/dprint/cache/plugins
[VERBOSE] Downloading url: https://plugins.dprint.dev/json-0.17.4.wasm
[VERBOSE] Creating instance of compiling
[VERBOSE] Created instance of compiling in 5ms
[VERBOSE] Dropped compiling (1 instances) in 0ms
[VERBOSE] Creating directory: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json
[VERBOSE] Writing file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.c2973d0f.tmp
[VERBOSE] Renaming /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.c2973d0f.tmp -> /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64
[VERBOSE] Writing file: /home/doox911/.cache/dprint/cache/plugin-cache-manifest.296480ae.tmp
[VERBOSE] Renaming /home/doox911/.cache/dprint/cache/plugin-cache-manifest.296480ae.tmp -> /home/doox911/.cache/dprint/cache/plugin-cache-manifest.json
[VERBOSE] Reading file: /home/doox911/.cache/dprint/cache/plugins/dprint-plugin-json/0.17.4-4.0.0-x86_64
[VERBOSE] Globbing: GlobPatterns { includes: Some([GlobPattern { relative_pattern: "**/*.{json,jsonc}", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_path]" } }]), excludes: [GlobPattern { relative_pattern: "!**/node_modules", base_dir: CanonicalizedPathBuf { path: "/mnt/c/Users/[my_path]" } }] }
[VERBOSE] File(s) matched: ["/mnt/c/Users/[my_path]/.prettierrc.json", "/mnt/c/Users/[my_path]/dprint.json", "/mnt/c/Users/[my_path]/large-file_1.json", "/mnt/c/Users/[my_path]/large-file_10.json", "/mnt/c/Users/[my_path]/large-file_100.json", "/mnt/c/Users/[my_path]/large-file_11.json", "/mnt/c/Users/[my_path]/large-file_12.json", "/mnt/c/Users/[my_path]/large-file_13.json", "/mnt/c/Users/[my_path]/large-file_14.json", "/mnt/c/Users/[my_path]/large-file_15.json", "/mnt/c/Users/[my_path]/large-file_16.json", "/mnt/c/Users/[my_path]/large-file_17.json", "/mnt/c/Users/[my_path]/large-file_18.json", "/mnt/c/Users/[my_path]/large-file_19.json", "/mnt/c/Users/[my_path]/large-file_2.json", "/mnt/c/Users/[my_path]/large-file_20.json", "/mnt/c/Users/[my_path]/large-file_21.json", "/mnt/c/Users/[my_path]/large-file_22.json", "/mnt/c/Users/[my_path]/large-file_23.json", "/mnt/c/Users/[my_path]/large-file_24.json", "/mnt/c/Users/[my_path]/large-file_25.json", "/mnt/c/Users/[my_path]/large-file_26.json", "/mnt/c/Users/[my_path]/large-file_27.json", "/mnt/c/Users/[my_path]/large-file_28.json", "/mnt/c/Users/[my_path]/large-file_29.json", "/mnt/c/Users/[my_path]/large-file_3.json", "/mnt/c/Users/[my_path]/large-file_30.json", "/mnt/c/Users/[my_path]/large-file_31.json", "/mnt/c/Users/[my_path]/large-file_32.json", "/mnt/c/Users/[my_path]/large-file_33.json", "/mnt/c/Users/[my_path]/large-file_34.json", "/mnt/c/Users/[my_path]/large-file_35.json", "/mnt/c/Users/[my_path]/large-file_36.json", "/mnt/c/Users/[my_path]/large-file_37.json", "/mnt/c/Users/[my_path]/large-file_38.json", "/mnt/c/Users/[my_path]/large-file_39.json", "/mnt/c/Users/[my_path]/large-file_4.json", "/mnt/c/Users/[my_path]/large-file_40.json", "/mnt/c/Users/[my_path]/large-file_41.json", "/mnt/c/Users/[my_path]/large-file_42.json", "/mnt/c/Users/[my_path]/large-file_43.json", "/mnt/c/Users/[my_path]/large-file_44.json", "/mnt/c/Users/[my_path]/large-file_45.json", "/mnt/c/Users/[my_path]/large-file_46.json", "/mnt/c/Users/[my_path]/large-file_47.json", "/mnt/c/Users/[my_path]/large-file_48.json", "/mnt/c/Users/[my_path]/large-file_49.json", "/mnt/c/Users/[my_path]/large-file_5.json", "/mnt/c/Users/[my_path]/large-file_50.json", "/mnt/c/Users/[my_path]/large-file_51.json", "/mnt/c/Users/[my_path]/large-file_52.json", "/mnt/c/Users/[my_path]/large-file_53.json", "/mnt/c/Users/[my_path]/large-file_54.json", "/mnt/c/Users/[my_path]/large-file_55.json", "/mnt/c/Users/[my_path]/large-file_56.json", "/mnt/c/Users/[my_path]/large-file_57.json", "/mnt/c/Users/[my_path]/large-file_58.json", "/mnt/c/Users/[my_path]/large-file_59.json", "/mnt/c/Users/[my_path]/large-file_6.json", "/mnt/c/Users/[my_path]/large-file_60.json", "/mnt/c/Users/[my_path]/large-file_61.json", "/mnt/c/Users/[my_path]/large-file_62.json", "/mnt/c/Users/[my_path]/large-file_63.json", "/mnt/c/Users/[my_path]/large-file_64.json", "/mnt/c/Users/[my_path]/large-file_65.json", "/mnt/c/Users/[my_path]/large-file_66.json", "/mnt/c/Users/[my_path]/large-file_67.json", "/mnt/c/Users/[my_path]/large-file_68.json", "/mnt/c/Users/[my_path]/large-file_69.json", "/mnt/c/Users/[my_path]/large-file_7.json", "/mnt/c/Users/[my_path]/large-file_70.json", "/mnt/c/Users/[my_path]/large-file_71.json", "/mnt/c/Users/[my_path]/large-file_72.json", "/mnt/c/Users/[my_path]/large-file_73.json", "/mnt/c/Users/[my_path]/large-file_74.json", "/mnt/c/Users/[my_path]/large-file_75.json", "/mnt/c/Users/[my_path]/large-file_76.json", "/mnt/c/Users/[my_path]/large-file_77.json", "/mnt/c/Users/[my_path]/large-file_78.json", "/mnt/c/Users/[my_path]/large-file_79.json", "/mnt/c/Users/[my_path]/large-file_8.json", "/mnt/c/Users/[my_path]/large-file_80.json", "/mnt/c/Users/[my_path]/large-file_81.json", "/mnt/c/Users/[my_path]/large-file_82.json", "/mnt/c/Users/[my_path]/large-file_83.json", "/mnt/c/Users/[my_path]/large-file_84.json", "/mnt/c/Users/[my_path]/large-file_85.json", "/mnt/c/Users/[my_path]/large-file_86.json", "/mnt/c/Users/[my_path]/large-file_87.json", "/mnt/c/Users/[my_path]/large-file_88.json", "/mnt/c/Users/[my_path]/large-file_89.json", "/mnt/c/Users/[my_path]/large-file_9.json", "/mnt/c/Users/[my_path]/large-file_90.json", "/mnt/c/Users/[my_path]/large-file_91.json", "/mnt/c/Users/[my_path]/large-file_92.json", "/mnt/c/Users/[my_path]/large-file_93.json", "/mnt/c/Users/[my_path]/large-file_94.json", "/mnt/c/Users/[my_path]/large-file_95.json", "/mnt/c/Users/[my_path]/large-file_96.json", "/mnt/c/Users/[my_path]/large-file_97.json", "/mnt/c/Users/[my_path]/large-file_98.json", "/mnt/c/Users/[my_path]/large-file_99.json", "/mnt/c/Users/[my_path]/package.json"]
[VERBOSE] Finished globbing in 16ms
[VERBOSE] Max threads: 8
Thread count: 7
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 1ms
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_12.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_1.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/.prettierrc.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/dprint.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_11.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_100.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_10.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 2ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/dprint.json in 3ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/.prettierrc.json in 2ms
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_14.json
[VERBOSE] Reading file: /mnt/c/Users/[my_path]/large-file_13.json
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 2ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 2ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] Creating instance of dprint-plugin-json
[VERBOSE] Created instance of dprint-plugin-json in 0ms
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_13.json in 7134ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_13.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_100.json in 7286ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_100.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_1.json in 7363ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_1.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_10.json in 7369ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_10.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_11.json in 7383ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_11.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_12.json in 7593ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_12.json
[VERBOSE] Formatted file: /mnt/c/Users/[my_path]/large-file_14.json in 7626ms
[VERBOSE] Ensuring stable format: /mnt/c/Users/[my_path]/large-file_14.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_1.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_11.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_12.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_100.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_10.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_14.json
WARNING: Formatting is slow for /mnt/c/Users/[my_path]/large-file_13.json
17109
17
0
dsherret commented 8 months ago

There were some memory improvements made in 0.19.2. Not sure if that fixes it, but maybe.

I still haven't been able to reproduce this one. Going to dump the test script I've been using that sets this up so I don't lose it again:

import $ from "jsr:@david/dax@0.39";

const data = await $.request("https://raw.githubusercontent.com/Nilstrieb/jsonformat/master/benches/large-file.json")
  .text();
const tempDir = $.path("tempDir");
tempDir.ensureDirSync();

for (let i = 0; i < 100; i++) {
  tempDir.join("file" + i + ".json").writeTextSync(data);
}

await $`dprint fmt --incremental=false`;