Hot reload not working on Windows 10.

[ZeroCool940711]

Describe the bug It seems like hot reloading is not working on Windows 10 when using the reload=True on a Flow or executor as well as when using it on the cli when running jina flow --uses flow.yml --reload. I went and created a simple new flow with jina new test-flow and then launched the flow with jina flow --uses flow.yml --reload, after modifying any of the files on the sub folders including the yml file for the flow and executor as well as the script for the executor I was expecting the server to be reloaded and it would take the new changes but nothing happened, only when manually restarting the jina server was that the new changes took effect.

---

Environment

- jina 3.17.0
- docarray 0.21.0
- jcloud 0.2.12
- jina-hubble-sdk 0.38.0
- jina-proto 0.1.27
- protobuf 4.23.3
- proto-backend upb
- grpcio 1.47.5
- pyyaml 6.0
- python 3.10.11
- platform Windows
- platform-release 10
- platform-version 10.0.19045
- architecture AMD64
- processor Intel64 Family 6 Model 44 Stepping 2, GenuineIntel
- uid 203050741247743
- session-id aa724427-0de7-11ee-88b0-b8ac6f4daeff
- uptime 2023-06-18T07:51:53.132540
- ci-vendor (unset)
- internal False
* JINA_DEFAULT_HOST (unset)
* JINA_DEFAULT_TIMEOUT_CTRL (unset)
* JINA_DEPLOYMENT_NAME (unset)
* JINA_DISABLE_UVLOOP (unset)
* JINA_EARLY_STOP (unset)
* JINA_FULL_CLI (unset)
* JINA_GATEWAY_IMAGE (unset)
* JINA_GRPC_RECV_BYTES (unset)
* JINA_GRPC_SEND_BYTES (unset)
* JINA_HUB_NO_IMAGE_REBUILD (unset)
* JINA_LOG_CONFIG (unset)
* JINA_LOG_LEVEL (DEBUG)
* JINA_LOG_NO_COLOR (unset)
* JINA_MP_START_METHOD (unset)
* JINA_OPTOUT_TELEMETRY (unset)
* JINA_RANDOM_PORT_MAX (unset)
* JINA_RANDOM_PORT_MIN (unset)
* JINA_LOCKS_ROOT (unset)
* JINA_K8S_ACCESS_MODES (unset)
* JINA_K8S_STORAGE_CLASS_NAME (unset)
* JINA_K8S_STORAGE_CAPACITY (unset)
* JINA_STREAMER_ARGS (unset)

[JoanFM]

Can you give a minimal reproducible example? an example of a folder structure and flow.yml that does not work? together with the logs when JINA_LOG_LEVEL=DEBUG

[ZeroCool940711]

Here is a ".zip" file containing a minimal example, you can launch the flow with jina flow --uses flow.yml --reload or launch it with python main.py, then modify the executor or any file inside, it should reload the server but it doesn't happen for me on Windows 10. jina-test.zip

[JoanFM]

what do u get when running it with JINA_LOG_LEVEL=DEBUG active and the reload=True put in the Flow.yml at the executor level? Then try to edit the file executor.py

jtype: Flow
version: '1'
gateway:
  protocol: [grpc, http, websocket]
  port: [54321, 54322, 54323]
executors:
  - uses: executor1/config.yml
    name: toyExecutor
    reload: True

[JoanFM]

Also, can you please configure how does it change when run from command line and when run with python main.py?

[ZeroCool940711]

It seems to be working when using reload=True only on the individual executors but not when using it on the flow itself, both running it thru the cli with the jina command or using it on a python script like the main.py I provided before for testing. I think using it on the whole flow should also work, that way we can add new executors and code without having to restart the whole jina server every time. Another interesting thing I noticed is that when launching it thru the jina command every time I modify the executor I get an error message telling me something is wrong with the executor even tho it first shows an INFO message saying everything is okay and that it reloaded the executor but when running the main.py script and doing the exact edit we only get the INFO message and not the ERROR one.

INFO   toyExecutor/rep-0@16948 detected changes in:                                             [06/19/23 01:20:54]
       ['D:\\Alejandro\\Projects\\Python\\AI\\ImageGen\\Muse\\muse-jina-trainer\\executor1\\ex…
       Refreshing the Executor
ERROR  toyExecutor/rep-0@16948  Exception when refreshing Executor when changes detected in
       ['D:\\Alejandro\\Projects\\Python\\AI\\ImageGen\\Muse\\muse-jina-trainer\\executor1\\ex…

[JoanFM]

Can you try running it on this branch? https://github.com/jina-ai/jina/pull/5915

I think there are more informative error message

[JoanFM]

What I observe on Linux is:

When using jina flow ..., there is an Exception when loading the Executor because of some module spec.

When using python main.py changing the Executor is working properly.

[JoanFM]

Having a Reload at Flow level is quite a complex task, but we do not think that people would iterate so often on adding and removing Executors but rather spend more time iterating on a single Executor.

[ZeroCool940711]

Can you try running it on this branch? #5915

I think there are more informative error message

I went and downloaded the branch and installed it, then ran jina flow ... like before and I think I got pretty much the same log, you might spot something I missed there tho. log.txt

Having a Reload at Flow level is quite a complex task, but we do not think that people would iterate so often on adding and removing Executors but rather spend more time iterating on a single Executor.

I understand that reloading it at flow level is a complex task but I think this is really needed, hot reloading is really useful when you start making a new flow from scratch or when you are learning how jina.ai works, you will most likely spend a lot of time modifying the flow at the start either to learn how things work or to change the structure of what you are doing to fit what you need, while you will spend a big chunk of your time editing the Executors you will also spend a similar amount of time editing the flow, specially if you are starting to learn how jina works or if there were big changes or new features that you want to fix or try out.

[JoanFM]

but I want to see thr log when u have the reload active while running it with python main.py

[ZeroCool940711]

oh, my bad, didn't read properly the message where you said that.

Here you go

log.txt

[JoanFM]

Can you pull again from that branch and try again? In this case I believe the refreshing is working, it seems not to be working (also on Linux) when used from command line jina flow --uses flow.yml (some thing about the modules being imported differently).

It would be helpful to make sure that with python main.py reload works at Executor level

[ZeroCool940711]

Here is the log for what I get when running the main.py: log.txt and when using it with jina flow: log.txt

Another note, it seems like when using the jina flow or any jina command while inside a virtual environment either with venv or conda env it will sometimes get out of the current environment and run jina from the base environment instead, this might be what is causing some issues, I tried this on another computer to make sure it wasn't something specific on my machine and I got the same behavior, its not constant tho and seem to happen randomly.

[JoanFM]

So in the case running main.py it seems to be working, the Executor is refreshed right?

[JoanFM]

if u run from venv, maybe u can try running python -m jina ...

[ZeroCool940711]

So in the case running main.py it seems to be working, the Executor is refreshed right?

They both seem to be working now, the only difference is that when using jina flow it will also show an error after the INFO message, it will still reload the executor tho.

if u run from venv, maybe u can try running python -m jina ...

Yes, when using the jina commands with python -m jina it will stay in the venv, I just thought of mentioning the issue with it getting out of the venv just in case, I thought it was worth mentioning it.

[JoanFM]

Also, for the Flow reload, can u try adding the reload=True option at Flow level inside the yaml instead of doing jina flow --uses flow.yml --reload? Would that be working?

[ZeroCool940711]

Also, for the Flow reload, can u try adding the reload=True option at Flow level inside the yaml instead of doing jina flow --uses flow.yml --reload? Would that be working?

Using it at Flow level in the yml file seems to do nothing, the table shown with the options jina is using at the start seems to also show that reload is False even tho in the yml its set to True which means it is ignoring the option set in there. log.txt

[ZeroCool940711]

Also, you might have noticed this from the logs Ive sent but sometimes the ip shown on the addresses for the endpoints when running the jina server seem to switch between ipv4 and ipv6, I think having an option to force which one to use would be good or just defaulting to ipv4 would be better.

[JoanFM]

This is weird, as per our code, we are checking the value returned by any of these services:

    'https://api.ipify.org',
    'https://ident.me',
    'https://checkip.amazonaws.com/'

What do they provide in ur case?

[ZeroCool940711]

This is weird, as per our code, we are checking the value returned by any of these services:

    'https://api.ipify.org',
    'https://ident.me',
    'https://checkip.amazonaws.com/'

What do they provide in ur case?

Check the log.txt files I sent before for specifics but sometimes I get my external IPv4 address but also sometimes I get my IPv6 such as 🌍 Public 2600:8800:3200:cc00:91ab:4ae1:e20:d6e0:54322, usually for the Public endpoints for each protocol, I think it also changes depending on whether I run things through a main script or if I use the jina flow command.

[JoanFM]

Here is the status of the investigation:

What is working?

Flow:

When u set reload flag to Flow yaml as this:

jtype: Flow
version: '1'
with:
  reload: True

Jina will watch changes inside the flow.yml and restart it when used both from command line and from main.py.

Deployment/Executor.

When u set the reload flag to a single Executor as:

jtype: Flow
version: '1'
with:
  reload: True
gateway:
  protocol: [grpc, http]
  port: [54321, 54322]
executors:
  - uses: executor1/config.yml
    name: toyExecutor
    reload: True

Jina will look for changes inside the executor1 folder to reload the single Executor, however this is not working when starting the flow from command line.