search

FS-Driverless / Formula-Student-Driverless-Simulator

A virtual world where Autonomous Systems from different Formula Student teams can compete in time-trial challenges
https://fs-driverless.github.io/Formula-Student-Driverless-Simulator
GNU General Public License v2.0
204 stars 85 forks source link

Cannot clone the repository (Git LFS is over its data quota) #366

Closed vrepetskyi closed 6 months ago

vrepetskyi commented 6 months ago 
git clone https://github.com/FS-Driverless/Formula-Student-Driverless-Simulator.git
Cloning into 'Formula-Student-Driverless-Simulator'...
remote: Enumerating objects: 8876, done.
remote: Counting objects: 100% (156/156), done.
remote: Compressing objects: 100% (48/48), done.
remote: Total 8876 (delta 63), reused 142 (delta 58), pack-reused 8720
Receiving objects: 100% (8876/8876), 688.11 MiB | 12.31 MiB/s, done.
Resolving deltas: 100% (4634/4634), done.
Downloading UE4Project/Content/Blocks/1M_Cube.uasset (85 KB)
Error downloading object: UE4Project/Content/Blocks/1M_Cube.uasset (3a702be): Smudge error: Error downloading UE4Project/Content/Blocks/1M_Cube.uasset (3a702be2ee121e56cb97f116be968e633774cd91d12207e43f7bc09f6383f428): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Errors logged to /home/vrepetskyi/classes/6/rob/Formula-Student-Driverless-Simulator/.git/lfs/logs/20240311T111249.002330546.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: UE4Project/Content/Blocks/1M_Cube.uasset: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'
wouter-heerwegh commented 6 months ago

Hi @vrepetskyi,

I just looked into it, it seems like we got 667 clones in just 1 day, even though there were 3 unique cloners on the 1st of March. image

This is much more than on a regular day: image

I'm not sure we can really do anything about that at the moment, but billing shows that the quota should reset today/tomorrow.

If anyone is running some testing scripts, please consider cloning the repo to a local directory and using your local copy to run tests.

vrepetskyi commented 6 months ago

Thanks for the quick response. I'll try again tomorrow then.

wouter-heerwegh commented 6 months ago

@vrepetskyi, bandwidth quota is reset, let me know if it works now. Feel free to close the issue if it is resolved.

vrepetskyi commented 6 months ago

Works now. Thanks again.

RafaGuil commented 5 months ago

Hi, I have the same issue:

` C:\Users\Rafa Guil Valero>git clone https://github.com/FS-Driverless/Formula-Student-Driverless-Simulator.git Cloning into 'Formula-Student-Driverless-Simulator'... remote: Enumerating objects: 8876, done. remote: Counting objects: 100% (156/156), done. remote: Compressing objects: 100% (48/48), done. remote: Total 8876 (delta 63), reused 142 (delta 58), pack-reused 8720Receiving objects: 100% (8876/8876), 682.89 MiB | Receiving objects: 100% (8876/8876), 688.11 MiB | 11.16 MiB/s, done.

Resolving deltas: 100% (4611/4611), done. Updating files: 100% (739/739), done. Downloading UE4Project/Content/Blocks/1M_Cube.uasset (85 KB) Error downloading object: UE4Project/Content/Blocks/1M_Cube.uasset (3a702be): Smudge error: Error downloading UE4Project/Content/Blocks/1M_Cube.uasset (3a702be2ee121e56cb97f116be968e633774cd91d12207e43f7bc09f6383f428): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Errors logged to 'C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator.git\lfs\logs\20240429T115759.0683178.log'. Use git lfs logs last to view the log. error: external filter 'git-lfs filter-process' failed fatal: UE4Project/Content/Blocks/1M_Cube.uasset: smudge filter lfs failed warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/'

C:\Users\Rafa Guil Valero>cd Formula-Student-Driverless-Simulator

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator>git lfs pull batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access. batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access. batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access. batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access. Failed to fetch some objects from 'https://github.com/FS-Driverless/Formula-Student-Driverless-Simulator.git/info/lfs' `

Could you reset the data quota of the repository? Thanks in advance

wouter-heerwegh commented 5 months ago

Hi @RafaGuil,

We're over the LFS data bandwith limit by 25GB this month. Quota is being reset today, this is not something I can manually reset, but is rather a monthly data limit . There is not much we can do about it except for paying 5$ more each month for more data. The annoying part is that clones from forks are also counting towards this repository, and there is no way to tell how much was used by each fork or to disable this.

If you're planning to just use the fsds_ros_bridge, you can clone without pulling LFS files:

GIT_LFS_SKIP_SMUDGE=1 git clone git@github.com:FS-Driverless/Formula-Student-Driverless-Simulator.git
RafaGuil commented 5 months ago

@wouter-heerwegh, I really need lfs files, I have to edit car's physics. What can I do to get those files? Is there any other way to get them?

wouter-heerwegh commented 5 months ago

@RafaGuil, as long as the data is not paid, we can't clone the LFS files anymore. I still have them locally, so as a temporary workaround, I'll upload them here.

@SijmenHuizenga, I see that all billing is forwarded towards you. Are you directly paying the 5 euro a month or is this still payed by FSOnline? I saw that there was also no payment history for February and March.

If payments stop, I guess we'll have to move to a different solution than LFS.

SijmenHuizenga commented 5 months ago

@RafaGuil can you try again? Bandwidth should be reset now.

Github stopped sending invoices because: image Re-submitting organization details fixed it, invoicing should continue.

Yes, I pay the datapacks. I would love to move to a free place to host git+lfs, but last time i checked (year ago) I couldn't find any. Suggestions are more than welcome.

RafaGuil commented 5 months ago

Yeah, now I have the lfs files Captura de pantalla de 2024-04-30 14-19-06

But I still have the same problem. When I open FSOnline.uproject I have this error: Captura de pantalla de 2024-04-30 14-43-30

wouter-heerwegh commented 5 months ago

@SijmenHuizenga, indeed, I also didn't find any nice alternate solutions. We could use a self hosted server, but it would be less friendly to use for teams.

@RafaGuil don't forget to compile AirSim. There are currently some issues when building the UE project on Ubuntu 20.04 and up, because of changing glibc versions and the toolchain of UE4.27. I would highly recommend working on the UE code in windows.

RafaGuil commented 5 months ago

@wouter-heerwegh Also I have errors in Windows...

Visual Studio 2019 Developer Command Prompt v16.11.35 Copyright (c) 2021 Microsoft Corporation

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community>cd ../../../..

C:>cd Users

C:\Users>cd "Rafa Guil Valero"

C:\Users\Rafa Guil Valero>cd Formula-Student-Driverless-Simulator

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator>cd AirSim

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>build.cmd

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>REM //---------- set up variable ----------

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>setlocal

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set ROOT_DIR=C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>REM // Check command line arguments

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set "noFullPolyCar="

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set "buildMode="

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>REM //check VS version

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>if "16.0" == "" ( echo( echo oh oh... You need to run this command from x64 Native Tools Command Prompt for VS 2019. goto :buildfailed_nomsg )

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>if "16.0" LSS "16.0" ( echo( echo Hello there! We just upgraded AirSim to Unreal Engine 4.24 and Visual Studio 2019. echo Here are few easy steps for upgrade so everything is new and shiny: echo https://github.com/Microsoft/AirSim/blob/master/docs/unreal_upgrade.md goto :buildfailed_nomsg )

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>if "" == "" goto noargs

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>chdir /d C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>REM //---------- Check cmake version ----------

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>CALL check_cmake.bat

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>REM //---------- set up variable ----------

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>setlocal

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set ROOT_DIR=C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set cmake_minversion_minmaj=" 3. 9"

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set "cmake_version= . "

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>where /q cmake

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>if 0 EQU 0 ( for /F "tokens=3" %a in ('cmake --version | find "cmake version"') do set cmake_version=%a if " . " == "" ( echo Unable to get version of cmake. 1>&2 exit /b 2 ) ) else ( echo cmake was not found in path. goto :download_install )

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set cmake_version=3.9.0

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set cmake_ver_major=

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set cmake_ver_minor=

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>for /F "tokens=1,2 delims=." %a in ("3.9.0") do ( set "cmake_ver_major= %a" set "cmake_ver_minor= %b" )

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>( set "cmake_ver_major= 3" set "cmake_ver_minor= 9" )

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>set cmake_ver_minmaj=" 3. 9"

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>if " 3. 9" LSS " 3. 9" ( echo Newer AirSim requires cmake verion " 3. 9" but you have " 3. 9" which is older. 1>&2 goto :download_install )

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>echo Found cmake version: 3.9.0 Found cmake version: 3.9.0

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>exit /b 0

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>if ERRORLEVEL 1 ( CALL check_cmake.bat if ERRORLEVEL 1 ( echo( echo ERROR: cmake was not installed correctly, we tried. goto :buildfailed ) )

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>REM //---------- Build rpclib ------------

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>ECHO Starting cmake to build rpclib... Starting cmake to build rpclib...

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>IF NOT EXIST external\rpclib\build mkdir external\rpclib\build

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>cd external\rpclib\build

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\external\rpclib\build>REM cmake -G"Visual Studio 14 2015 Win64" ..

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\external\rpclib\build>cmake -G"Visual Studio 16 2019" .. CMake Error: Could not create named generator Visual Studio 16 2019

Generators Visual Studio 15 2017 [arch] = Generates Visual Studio 2017 project files. Optional [arch] can be "Win64" or "ARM". Visual Studio 14 2015 [arch] = Generates Visual Studio 2015 project files. Optional [arch] can be "Win64" or "ARM". Visual Studio 12 2013 [arch] = Generates Visual Studio 2013 project files. Optional [arch] can be "Win64" or "ARM". Visual Studio 11 2012 [arch] = Generates Visual Studio 2012 project files. Optional [arch] can be "Win64" or "ARM". Visual Studio 10 2010 [arch] = Generates Visual Studio 2010 project files. Optional [arch] can be "Win64" or "IA64". Visual Studio 9 2008 [arch] = Generates Visual Studio 2008 project files. Optional [arch] can be "Win64" or "IA64". Visual Studio 8 2005 [arch] = Deprecated. Generates Visual Studio 2005 project files. Optional [arch] can be "Win64". Borland Makefiles = Generates Borland makefiles. NMake Makefiles = Generates NMake makefiles. NMake Makefiles JOM = Generates JOM makefiles. Green Hills MULTI = Generates Green Hills MULTI files (experimental, work-in-progress). MSYS Makefiles = Generates MSYS makefiles. MinGW Makefiles = Generates a make file for use with mingw32-make. Unix Makefiles = Generates standard UNIX makefiles. Ninja = Generates build.ninja files. Watcom WMake = Generates Watcom WMake makefiles. CodeBlocks - MinGW Makefiles = Generates CodeBlocks project files. CodeBlocks - NMake Makefiles = Generates CodeBlocks project files. CodeBlocks - NMake Makefiles JOM = Generates CodeBlocks project files. CodeBlocks - Ninja = Generates CodeBlocks project files. CodeBlocks - Unix Makefiles = Generates CodeBlocks project files. CodeLite - MinGW Makefiles = Generates CodeLite project files. CodeLite - NMake Makefiles = Generates CodeLite project files. CodeLite - Ninja = Generates CodeLite project files. CodeLite - Unix Makefiles = Generates CodeLite project files. Sublime Text 2 - MinGW Makefiles = Generates Sublime Text 2 project files. Sublime Text 2 - NMake Makefiles = Generates Sublime Text 2 project files. Sublime Text 2 - Ninja = Generates Sublime Text 2 project files. Sublime Text 2 - Unix Makefiles = Generates Sublime Text 2 project files. Kate - MinGW Makefiles = Generates Kate project files. Kate - NMake Makefiles = Generates Kate project files. Kate - Ninja = Generates Kate project files. Kate - Unix Makefiles = Generates Kate project files. Eclipse CDT4 - NMake Makefiles = Generates Eclipse CDT 4.0 project files. Eclipse CDT4 - MinGW Makefiles = Generates Eclipse CDT 4.0 project files. Eclipse CDT4 - Ninja = Generates Eclipse CDT 4.0 project files. Eclipse CDT4 - Unix Makefiles= Generates Eclipse CDT 4.0 project files.

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\external\rpclib\build>if "" == "--Debug" (cmake --build . --config Debug ) else if "" == "--Release" (cmake --build . --config Release ) else ( cmake --build . cmake --build . --config Release ) Error: could not load cache Error: could not load cache

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\external\rpclib\build>if ERRORLEVEL 1 goto :buildfailed

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\external\rpclib\build>echo(

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\external\rpclib\build>echo #### Build failed - see messages above. 1>&2

Build failed - see messages above.

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\external\rpclib\build>chdir /d C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim\

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>exit /b 1

C:\Users\Rafa Guil Valero\Formula-Student-Driverless-Simulator\AirSim>

I have Windows 11, UE 4.27 and VS 2019

wouter-heerwegh commented 5 months ago

@RafaGuil I'm moving your last comment to a new issue, as this is not related to the LFS issues.