JSON configs are not included in the built package

[Chaoses-Ib]

> git clone https://github.com/hiddenswitch/ComfyUI.git
> cd ComfyUI
> pip install --no-build-isolation .

> ls d:\venv\lib\site-packages\comfy\*.py

    Directory: D:\venv\lib\site-packages\comfy

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           2024-4-11    15:09              0 __init__.py
-a---           2024-4-11    15:09            299 checkpoint_pickle.py
...

> ls d:\venv\lib\site-packages\comfy\*.json
# empty

sd1_clip_config.json and other JSON configs are not installed. This leads to FileNotFoundError when executing:

Using xformers attention in VAE
!!! Exception during processing !!!
Traceback (most recent call last):
  File "d:\venv\lib\site-packages\comfy\cmd\execution.py", line 183, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "d:\venv\lib\site-packages\comfy\cmd\execution.py", line 102, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "d:\venv\lib\site-packages\comfy\cmd\execution.py", line 95, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "d:\venv\lib\site-packages\comfy\nodes\base_nodes.py", line 504, in load_checkpoint
    out = sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings"))
  File "d:\venv\lib\site-packages\comfy\sd.py", line 535, in load_checkpoint_guess_config
    clip = CLIP(clip_target, embedding_directory=embedding_directory)
  File "d:\venv\lib\site-packages\comfy\sd.py", line 103, in __init__
    self.cond_stage_model = clip(**(params))
  File "d:\venv\lib\site-packages\comfy\sd1_clip.py", line 519, in __init__
    setattr(self, self.clip, clip_model(device=device, dtype=dtype, **kwargs))
  File "d:\venv\lib\site-packages\comfy\sd1_clip.py", line 81, in __init__
    with open(textmodel_json_config) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'd:\\venv\\lib\\site-packages\\comfy\\sd1_clip_config.json'

Editable install works.

[doctorpangloss]

Thanks for the report. I could not reproduce this issue on Windows using a ReFS D:/ drive, please carefully look at my commands and see if there's anything I missed:

python -m venv test-venv
source test-venv/scripts/activate
git clone https://github.com/hiddenswitch/ComfyUI.git comfyui-test
cd comfyUI-test/
pip install wheel
pip install --no-build-isolation .
cd ..
ls test-venv/Lib/site-packages/comfy/*.json
pip install xformers==0.0.26.post1
mkdir workdir
comfyui -w workdir/ --listen 0.0.0.0 --port 9188

Results on Windows:

D:/ $ python --version
Python 3.11.7
D:/ $ python -m venv test-venv
D:/ $ source test-venv/scripts/activate
(test-venv) D:/ $ git clone https://github.com/hiddenswitch/ComfyUI.git comfyui-test
Cloning into 'comfyui-test'...
remote: Enumerating objects: 13011, done.
remote: Counting objects: 100% (234/234), done.
remote: Compressing objects: 100% (129/129), done.
remote: Total 13011 (delta 129), reused 165 (delta 104), pack-reused 12777
Receiving objects: 100% (13011/13011), 5.09 MiB | 8.39 MiB/s, done.
Resolving deltas: 100% (8793/8793), done.
(test-venv) D:/ $ cd comfyUI-test/
(test-venv) D:/comfyui-test $ pip install wheel
Collecting wheel
  Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata
  Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
Using cached wheel-0.43.0-py3-none-any.whl (65 kB)
Installing collected packages: wheel
Successfully installed wheel-0.43.0
(test-venv) D:/comfyui-test $ pip install --no-build-isolation .
Processing d:\comfyui-test
  Preparing metadata (pyproject.toml) ... done
Collecting torch@ https://download.pytorch.org/whl/cu121/torch-2.3.0%2Bcu121-cp311-cp311-win_amd64.whl#sha256=f7876ec20b42dd569e7a11c5af36febccc03830f63dfdedbd4026506e086cab6 (from comfyui==0.0.1)
...
(test-venv) D:/ $ ls test-venv/Lib/site-packages/comfy/*.json
test-venv/Lib/site-packages/comfy/clip_config_bigg.json         test-venv/Lib/site-packages/comfy/clip_vision_config_vitl.json
test-venv/Lib/site-packages/comfy/clip_vision_config_g.json     test-venv/Lib/site-packages/comfy/sd1_clip_config.json
test-venv/Lib/site-packages/comfy/clip_vision_config_h.json     test-venv/Lib/site-packages/comfy/sd2_clip_config.json
(test-venv) D:/ $ pip install xformers==0.0.26.post1
Collecting xformers==0.0.26.post1
...
Installing collected packages: xformers
Successfully installed xformers-0.0.26.post1
(test-venv) D:/ $ mkdir workdir
(test-venv) D:/ $ comfyui -w workdir/ --listen 0.0.0.0 --port 9188
Total VRAM 24564 MB, total RAM 130995 MB
xformers version: 0.0.26.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA RTX A5000 : native
VAE dtype: torch.bfloat16
Using xformers cross attention
Starting server

To see the GUI go to: http://localhost:9188
got prompt
D:\test-venv\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
v1-5-pruned-emaonly.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 4.27G/4.27G [00:40<00:00, 106MB/s]
model_type EPS
Using xformers attention in VAE
Using xformers attention in VAE
clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']
Requested to load SD1ClipModel
Loading 1 new model
Requested to load BaseModel
Loading 1 new model
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 10.01it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 46.82 seconds

This worked with both pip 23.2 and pip 24.0. Kindly provide any other insights or more reproduction details because I would like to fix this issue for you.

[Chaoses-Ib]

Updating setuptools from v57.4.0 to v69.5.1 solves the problem. It turns out that recursive globs (**) in package_data is first supported in v62.3.0 and doesn't work in earlier versions.

[doctorpangloss]

I can improve this since I like compatibility. I have set up Fridays as my regularly scheduled time to fix things. While I have you, please ticket anything that would make ComfyScript better for you.