search

adap / flower

Flower: A Friendly Federated Learning Framework
https://flower.ai
Apache License 2.0
4.71k stars 815 forks source link

Simulation with flwr==1.0.0 not working #1396

Open gubertoli opened 1 year ago

gubertoli commented 1 year ago

Describe the bug

Hi, I am trying to use the flwr simulation in the version 1.0.0 with Python 3.10.5. However, when I start the simulation with fl.simulation.start_simulation, it presents the following:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Input In [95], in <cell line: 8>()
      5 strategy=SaveModelStrategy()
      7 # Start simulation
----> 8 fl.simulation.start_simulation(
      9     client_fn=client_fn,
     10     num_clients=5,
     11     clients_ids=clients_list,
     12     num_rounds=NUM_ROUNDS,
     13     strategy=strategy,
     14 )

File ~\example\venv\lib\site-packages\flwr\simulation\__init__.py:33, in start_simulation(*args, **kwargs)
     31 def start_simulation(*args, **kwargs):  # type: ignore
     32     """Print error stating that ray is missing."""
---> 33     raise ImportError(RAY_IMPORT_ERROR)

ImportError: Unable to import module `ray`.

To install the necessary dependencies, install `flwr` with the `simulation` extra:

    pip install -U flwr["simulation"]

If I try to install the flwr with simulation extra as recommended, it downgrades to 0.16.0:

Requirement already satisfied: flwr[simulation] in c:\users\gustavo\example\venv\lib\site-packages (0.16.0)
Collecting flwr[simulation]
  Using cached flwr-1.0.0-py3-none-any.whl (90 kB)
Requirement already satisfied: numpy<2.0.0,>=1.21.0 in c:\users\gustavo\example\venv\lib\site-packages (from flwr[simulation]) (1.23.2)
Requirement already satisfied: protobuf<4.0.0,>=3.19.0 in c:\users\gustavo\example\venv\lib\site-packages (from flwr[simulation]) (3.19.4)
Requirement already satisfied: iterators<0.0.3,>=0.0.2 in c:\users\gustavo\example\venv\lib\site-packages (from flwr[simulation]) (0.0.2)
Requirement already satisfied: grpcio<2.0.0,>=1.43.0 in c:\users\gustavo\example\venv\lib\site-packages (from flwr[simulation]) (1.47.0)
  Using cached flwr-0.19.0-py3-none-any.whl (106 kB)
  Using cached flwr-0.18.0-py3-none-any.whl (106 kB)
Collecting grpcio<=1.43.0,>=1.27.2
  Using cached grpcio-1.43.0-cp310-cp310-win_amd64.whl (3.4 MB)
Requirement already satisfied: google<3.0.0,>=2.0.3 in c:\users\gustavo\example\venv\lib\site-packages (from flwr[simulation]) (2.0.3)
Collecting flwr[simulation]
  Using cached flwr-0.17.0-py3-none-any.whl (229 kB)
Requirement already satisfied: beautifulsoup4 in c:\users\gustavo\example\venv\lib\site-packages (from google<3.0.0,>=2.0.3->flwr[simulation]) (4.11.1)
Requirement already satisfied: six>=1.5.2 in c:\users\gustavo\example\venv\lib\site-packages (from grpcio<2.0.0,>=1.43.0->flwr[simulation]) (1.16.0)
Requirement already satisfied: soupsieve>1.2 in c:\users\gustavo\example\venv\lib\site-packages (from beautifulsoup4->google<3.0.0,>=2.0.3->flwr[simulation]) (2.3.2.post1)
WARNING: flwr 0.16.0 does not provide the extra 'simulation'
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the 'C:\Users\gustavo\example\venv\Scripts\python.exe -m pip install --upgrade pip' command.

I tried the following without success: pip install -U flwr["simulation"]==1.0.0

Steps/Code to Reproduce

pip install -U flwr["simulation"]

Expected Results

Flower with simulation capabilities.

Actual Results

Not able to run simulations with flower 1.0.0.

danieljanes commented 1 year ago

Thanks for the the detailed report @gubertoli, does it change anything if you do a pip install -U flwr[simulation] (note the missing ")?

The Flower tutorial installs 1.0 on Google Colab, but it's running on Python 3.7

davideNasi commented 1 year ago

Did you try changing "num_rounds=NUM_ROUNDS," to "config=fl.server.ServerConfig(num_rounds=NUM_ROUNDS)," ?

gubertoli commented 1 year ago

Thanks for the the detailed report @gubertoli, does it change anything if you do a pip install -U flwr[simulation] (note the missing ")?

The Flower tutorial installs 1.0 on Google Colab, but it's running on Python 3.7

It is going to flwr 0.16.0 also without ", follows an output sample:

(venv) C:\Users\gustavo\example-nids-fl>pip install -U flwr[simulation]
Requirement already satisfied: flwr[simulation] in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (1.0.0)
Requirement already satisfied: iterators<0.0.3,>=0.0.2 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (0.0.2)
Requirement already satisfied: numpy<2.0.0,>=1.21.0 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (1.23.2)
Requirement already satisfied: grpcio<2.0.0,>=1.43.0 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (1.47.0)
Requirement already satisfied: protobuf<4.0.0,>=3.19.0 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (3.19.4)
Collecting flwr[simulation]
  Using cached flwr-1.0.0-py3-none-any.whl (90 kB)
  Using cached flwr-0.19.0-py3-none-any.whl (106 kB)
  Using cached flwr-0.18.0-py3-none-any.whl (106 kB)
Collecting grpcio<=1.43.0,>=1.27.2
  Using cached grpcio-1.43.0-cp310-cp310-win_amd64.whl (3.4 MB)
Requirement already satisfied: google<3.0.0,>=2.0.3 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (2.0.3)
Collecting flwr[simulation]
  Using cached flwr-0.17.0-py3-none-any.whl (229 kB)
  Using cached flwr-0.16.0-py3-none-any.whl (216 kB)
WARNING: flwr 0.16.0 does not provide the extra 'simulation'
Requirement already satisfied: beautifulsoup4 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from google<3.0.0,>=2.0.3->flwr[simulation]) (4.11.1)
Requirement already satisfied: six>=1.5.2 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from grpcio<2.0.0,>=1.43.0->flwr[simulation]) (1.16.0)
Requirement already satisfied: soupsieve>1.2 in c:\users\gustavo\example-nids-fl\venv\lib\site-packages (from beautifulsoup4->google<3.0.0,>=2.0.3->flwr[simulation]) (2.3.2.post1)
Installing collected packages: flwr
  Attempting uninstall: flwr
    Found existing installation: flwr 1.0.0
    Uninstalling flwr-1.0.0:
      Successfully uninstalled flwr-1.0.0
Successfully installed flwr-0.16.0
gubertoli commented 1 year ago

Did you try changing "num_rounds=NUM_ROUNDS," to "config=fl.server.ServerConfig(num_rounds=NUM_ROUNDS)," ?

Good point, I just checked the changes for simulation from 0.19.0 to 1.0.0 here. However, even complying with the new syntax, still outputs:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Input In [29], in <cell line: 13>()
      4 strategy=fl.server.strategy.FedAvg(
      5         #fraction_fit=0.1,  # Sample 10% of available clients for training
      6         #fraction_evaluate=0.05,  # Sample 5% of available clients for evaluation
   (...)
      9         #min_available_clients=int(NUM_CLIENTS * 0.75),  # Wait until at least 75 clients are available
     10 )
     12 # Start simulation
---> 13 fl.simulation.start_simulation(
     14     client_fn=client_fn,
     15     num_clients=NUM_CLIENTS,
     16     config=fl.server.ServerConfig(num_rounds=5),
     17     strategy=strategy,
     18 )

File ~\example-nids-fl\venv\lib\site-packages\flwr\simulation\__init__.py:33, in start_simulation(*args, **kwargs)
     31 def start_simulation(*args, **kwargs):  # type: ignore
     32     """Print error stating that ray is missing."""
---> 33     raise ImportError(RAY_IMPORT_ERROR)

ImportError: Unable to import module `ray`.

To install the necessary dependencies, install `flwr` with the `simulation` extra:

    pip install -U flwr["simulation"]
shg550 commented 1 year ago

I have the same issue and my python version is Python 3.10.4. Could it be because of the python version?

danieljanes commented 1 year ago

@shg550 @gubertoli could your try this again with Flower 1.1 (which was released two days ago)? The ray dependency has been updated, so it might work better with more recent versions of Python

gubertoli commented 1 year ago

@danieljanes just tested with Flower 1.1 and the problem still happens:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Input In [26], in <cell line: 13>()
      4 strategy=fl.server.strategy.FedAvg(
      5         #fraction_fit=0.1,  # Sample 10% of available clients for training
      6         #fraction_evaluate=0.05,  # Sample 5% of available clients for evaluation
   (...)
      9         #min_available_clients=int(NUM_CLIENTS * 0.75),  # Wait until at least 75 clients are available
     10 )
     12 # Start simulation
---> 13 fl.simulation.start_simulation(
     14     client_fn=client_fn,
     15     num_clients=NUM_CLIENTS,
     16     config=fl.server.ServerConfig(num_rounds=5),
     17     strategy=strategy,
     18 )

File ~\Downloads\example-nids-fl\venv\lib\site-packages\flwr\simulation\__init__.py:33, in start_simulation(*args, **kwargs)
     31 def start_simulation(*args, **kwargs):  # type: ignore
     32     """Log error stating that module `ray` could not be imported."""
---> 33     raise ImportError(RAY_IMPORT_ERROR)

ImportError: Unable to import module `ray`.

To install the necessary dependencies, install `flwr` with the `simulation` extra:

    pip install -U flwr["simulation"]

If I run pip install -U flwr["simulation"], then it downgrades to 0.16.0, as follows:

Requirement already satisfied: flwr[simulation] in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (1.1.0)
Requirement already satisfied: numpy<2.0.0,>=1.21.0 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (1.23.2)
Requirement already satisfied: protobuf<4.0.0,>=3.19.0 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (3.19.4)
Requirement already satisfied: grpcio<2.0.0,>=1.43.0 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (1.47.0)
Requirement already satisfied: iterators<0.0.3,>=0.0.2 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (0.0.2)
Collecting flwr[simulation]
  Using cached flwr-1.1.0-py3-none-any.whl (121 kB)
  Using cached flwr-1.0.0-py3-none-any.whl (90 kB)
  Using cached flwr-0.19.0-py3-none-any.whl (106 kB)
  Using cached flwr-0.18.0-py3-none-any.whl (106 kB)
Requirement already satisfied: google<3.0.0,>=2.0.3 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from flwr[simulation]) (2.0.3)
Collecting grpcio<=1.43.0,>=1.27.2
  Using cached grpcio-1.43.0-cp310-cp310-win_amd64.whl (3.4 MB)
Collecting flwr[simulation]
  Using cached flwr-0.17.0-py3-none-any.whl (229 kB)
  Using cached flwr-0.16.0-py3-none-any.whl (216 kB)
Requirement already satisfied: beautifulsoup4 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from google<3.0.0,>=2.0.3->flwr[simulation]) (4.11.1)
Requirement already satisfied: six>=1.5.2 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from grpcio<2.0.0,>=1.43.0->flwr[simulation]) (1.16.0)
Requirement already satisfied: soupsieve>1.2 in c:\users\gustavo\downloads\example-nids-fl\venv\lib\site-packages (from beautifulsoup4->google<3.0.0,>=2.0.3->flwr[simulation]) (2.3.2.post1)
Installing collected packages: flwr
  Attempting uninstall: flwr
    Found existing installation: flwr 1.1.0
    Uninstalling flwr-1.1.0:
      Successfully uninstalled flwr-1.1.0
Successfully installed flwr-0.16.0
WARNING: flwr 0.16.0 does not provide the extra 'simulation'
WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
You should consider upgrading via the 'C:\Users\gustavo\Downloads\example-nids-fl\venv\Scripts\python.exe -m pip install --upgrade pip' command.

The setup in use:

Packages:

joselsalmeron commented 1 year ago

I am still getting the same issue. My machine is a M2 chip, with python==1.14 and flower==1.4.0. Nothing changes when I tried the installation pip install -U flwr["simulation"] and pip install -U flwr[simulation].

Happy2Git commented 1 year ago

same issue on my M1 Mac.

``` (FedLe) ➜ femnist_baseline git:(main) ✗ python -m pip install -U --pre "flwr[simulation]" Requirement already satisfied: flwr[simulation] in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (1.4.0) Requirement already satisfied: grpcio!=1.52.0,<2.0.0,>=1.48.2 in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (from flwr[simulation]) (1.56.0) Requirement already satisfied: iterators<0.0.3,>=0.0.2 in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (from flwr[simulation]) (0.0.2) Requirement already satisfied: numpy<2.0.0,>=1.21.0 in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (from flwr[simulation]) (1.25.0) Requirement already satisfied: protobuf<4.0.0,>=3.19.0 in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (from flwr[simulation]) (3.20.3) INFO: pip is looking at multiple versions of flwr[simulation] to determine which version is compatible with other requirements. This could take a while. Collecting flwr[simulation] Using cached flwr-1.4.0-py3-none-any.whl (157 kB) Using cached flwr-1.3.0-py3-none-any.whl (139 kB) Using cached flwr-1.2.0-py3-none-any.whl (133 kB) Using cached flwr-1.1.0-py3-none-any.whl (121 kB) INFO: pip is looking at multiple versions of flwr[simulation] to determine which version is compatible with other requirements. This could take a while. Using cached flwr-1.0.0-py3-none-any.whl (90 kB) Downloading flwr-1.0.0rc0-py3-none-any.whl (90 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.1/90.1 kB 650.5 kB/s eta 0:00:00 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C. Downloading flwr-1.0.0a0-py3-none-any.whl (90 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.4/90.4 kB 2.6 MB/s eta 0:00:00 Using cached flwr-0.19.0-py3-none-any.whl (106 kB) Using cached flwr-0.18.0-py3-none-any.whl (106 kB) Requirement already satisfied: google<3.0.0,>=2.0.3 in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (from flwr[simulation]) (2.0.3) Collecting grpcio<=1.43.0,>=1.27.2 (from flwr[simulation]) Using cached grpcio-1.43.0.tar.gz (21.5 MB) Preparing metadata (setup.py) ... done Collecting flwr[simulation] Using cached flwr-0.17.0-py3-none-any.whl (229 kB) Using cached flwr-0.16.0-py3-none-any.whl (216 kB) WARNING: flwr 0.16.0 does not provide the extra 'simulation' Requirement already satisfied: beautifulsoup4 in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (from google<3.0.0,>=2.0.3->flwr[simulation]) (4.12.2) Requirement already satisfied: soupsieve>1.2 in /Users/Happy/mambaforge/envs/FedLe/lib/python3.11/site-packages (from beautifulsoup4->google<3.0.0,>=2.0.3->flwr[simulation]) (2.4.1) Installing collected packages: flwr Attempting uninstall: flwr Found existing installation: flwr 1.4.0 Uninstalling flwr-1.4.0: Successfully uninstalled flwr-1.4.0 Successfully installed flwr-0.16.0 ```

But after I installed ray pip install -U "ray[default] @ LINK_TO_WHEEL.whl" by following https://docs.ray.io/en/latest/ray-overview/installation.html, I successfully run the flwr.simulation without installing flwr[simulation]. @joselsalmeron

takumiSudo commented 10 months ago

I was having the same issue for my M1 mac.

Followed the steps @Happy2Git provided, and the process went smoothly! Thanks

gubertoli commented 4 months ago

I had this same issue with flwr 1.7.0 and Python 3.12 (trying pip install flwr[simulation] downgrades to flwr==0.16.0). To solve, I had to downgrade to Python 3.10.

Seems to be a problem between the Python version and Ray: https://discuss.ray.io/t/installing-ray-in-python-3-11-2/9431/2