Is sourcing `$IDF_PATH/export.sh` in `xs-dev-export.sh` file still necessary? (ESP32 setup)

[achou11]

After running xs-dev setup --device esp32, I see the following line in the xs-dev-export.sh script (which is then sourced by my bashrc file):

source $IDF_PATH/export.sh 1> /dev/null

Wondering if that's necessary to include based on https://github.com/HipsterBrown/xs-dev/issues/96#issuecomment-1400825571? Main reason I ask is because the output of that script isn't being silenced when I open a new integrated terminal in VSCode (and only VSCode, strangely...), so I end up seeing output like the following each time I open a new terminal instance there:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'click'
bash: [: : integer expression expected
Cannot import module "click". This usually means that "idf.py" was not spawned within an ESP-IDF shell environment or the python virtual environment used by "idf.py" is corrupted.
Please use idf.py only in an ESP-IDF shell environment. If problem persists, please try to install ESP-IDF tools again as described in the Get Started guide.

Basically wondering if it's okay to just comment it out in my case, where the Moddable SDK version supports running that if needed automatically?

---

Env info:

xs-dev environment info:
  CLI Version                0.30.3                                                                                   
  OS                         Darwin                                                                                   
  Arch                       x64                                                                                      
  Shell                      /bin/bash                                                                                
  NodeJS Version             v20.11.0 (/Users/andrewchou/Library/Caches/fnm_multishells/55110_1706385419215/bin/node) 
  Python Version             3.12.1 (/Users/andrewchou/.local/share/mise/installs/python/latest/bin/python)           
  Moddable SDK Version       4.3.8 (/Users/andrewchou/.local/share/moddable)                                          
  Supported target devices   mac, esp32                                                                               
  ESP32 IDF Directory        /Users/andrewchou/.local/share/esp32/esp-idf 

[HipsterBrown]

Thanks for asking about this! xs-dev does need to know about esptool.py directly in order to scan for devices, however it might not need the whole export from the ESP-IDF in order to call that CLI.

Definitely worth exploring. 🤔

[achou11]

thanks for the explanation! to be clear, I don't think there's any issue on xs-dev's side - I just thought there might be because of the error output I was seeing. Outside of VSCode terminals it's typically just silenced.

Guessing this is more of an issue with esp-idf and/or VSCode.

But yeah if this helps consider the usage of that export script, that seems like a good outcome 😄 especially since it seems to add a little bit of overhead for general shell initialization time.

maybe https://github.com/espressif/esp-idf/issues/10926 is relevant??

[achou11]

Just looking through their setup docs (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linux-macos-setup.html#step-4-set-up-the-environment-variables) and noticed this note:

If you plan to use esp-idf frequently, you can create an alias for executing export.sh:

    1. Copy and paste the following command to your shell's profile (.profile, .bashrc, .zprofile, etc.)

    alias get_idf='. $HOME/esp/esp-idf/export.sh'

    2. Refresh the configuration by restarting the terminal session or by running source [path to profile], for example, source ~/.bashrc.

Now you can run get_idf to set up or refresh the esp-idf environment in any terminal session.

Technically, you can add export.sh to your shell's profile directly; however, it is not recommended. Doing so activates IDF virtual environment in every terminal session (including those where IDF is not needed), defeating the purpose of the virtual environment and likely affecting other software.

Seems like calling the script as part of the shell profile (or rc) is NOT recommended since it's supposed to be done more intentionally (note that they recommend adding an alias that you may export as part of the rc file, but not running the actual script).

Thoughts?

[phoddie]

FWIW - That particular error (click missing) usually indicates a Python version mismatch. If ESP-IDF works after that you are very lucky.

[achou11]

FWIW - That particular error (click missing) usually indicates a Python version mismatch. If ESP-IDF works after that you are very lucky.

ah good to know! can you elaborate about what kind of mismatch? i.e. between which tools, or what's the expected??

[achou11]

Update: restarted VSCode and seems like everything is working as expected 🤦 Before I was just starting a new terminal session in the same VSCode session so maybe that wasn't enough of a full reset for some reason...

Can confirm that the output of the export.sh is being silenced now. Just to check the output and whether I was getting the click module error, I commented out the 1> /dev/null part and have the following output:

Detecting the Python interpreter
Checking "python3" ...
Python 3.12.1
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Constraint file: /Users/andrewchou/.espressif/espidf.constraints.v5.1.txt
Requirement files:
 - /Users/andrewchou/.local/share/esp32/esp-idf/tools/requirements/requirements.core.txt
Python being checked: /Users/andrewchou/.espressif/python_env/idf5.1_py3.12_env/bin/python
Python requirements are satisfied.
Added the following directories to PATH:
  /Users/andrewchou/.local/share/esp32/esp-idf/components/espcoredump
  /Users/andrewchou/.local/share/esp32/esp-idf/components/partition_table
  /Users/andrewchou/.local/share/esp32/esp-idf/components/app_update
  /Users/andrewchou/.espressif/tools/xtensa-esp-elf-gdb/12.1_20221002/xtensa-esp-elf-gdb/bin
  /Users/andrewchou/.espressif/tools/riscv32-esp-elf-gdb/12.1_20221002/riscv32-esp-elf-gdb/bin
  /Users/andrewchou/.espressif/tools/xtensa-esp32-elf/esp-12.2.0_20230208/xtensa-esp32-elf/bin
  /Users/andrewchou/.espressif/tools/xtensa-esp32s2-elf/esp-12.2.0_20230208/xtensa-esp32s2-elf/bin
  /Users/andrewchou/.espressif/tools/xtensa-esp32s3-elf/esp-12.2.0_20230208/xtensa-esp32s3-elf/bin
  /Users/andrewchou/.espressif/tools/riscv32-esp-elf/esp-12.2.0_20230208/riscv32-esp-elf/bin
  /Users/andrewchou/.espressif/tools/esp32ulp-elf/2.35_20220830/esp32ulp-elf/bin
  /Users/andrewchou/.espressif/tools/openocd-esp32/v0.12.0-esp32-20230921/openocd-esp32/bin
  /Users/andrewchou/.espressif/python_env/idf5.1_py3.12_env/bin
  /Users/andrewchou/.local/share/esp32/esp-idf/tools
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

My guess is that this looks good and expected and I'm not going to have an issue moving forward (famous last words).

Apologies for the incredible amount of misdirection in this issue as a whole 😅 Would still reconsider whether adding that export script as something that's sourced in a rc file makes sense, just because it does take a perceptible amount of time to run (and affects all terminal sessions, not just for ESP-related dev)

[phoddie]

can you elaborate about what kind of mismatch? i.e. between which tools, or what's the expected??

The details are hazy. My memory is that it happens when Python 3.x is expected but 2.x is found. ESP-IDF carries its own copy of Python to avoid this sort of thing, but it doesn't always seem to work...

I agree that it is optimal to only source the ESP-IDF when it is needed. That's why I changed the Moddable SDK to do that. ;) Something similar should be possible with xs-dev. I defer to @HipsterBrown on how complex that may be to achieve.

[achou11]

The details are hazy. My memory is that it happens when Python 3.x is expected but 2.x is found. ESP-IDF carries its own copy of Python to avoid this sort of thing, but it doesn't always seem to work...

Got it! Think the issue went away for me based on what I described in https://github.com/HipsterBrown/xs-dev/issues/161#issuecomment-1913392967, so fingers crossed 🤞