Can't install Khoj Backend

flowingblaze commented 11 months ago

Hello, I am on windows 10 and installed the khoj plugin for obsidian because I wanted to try it out but it gave an error saying I need to install the backend. So after looking at the docs I ran py -m pip install khoj-assistant as instructed. I then got the following error: "ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\Python311\share' Consider using the --user option or check the permissions." so I did the same command as before but with --user at the end as instructed and I tried to run khoj via typing "khoj" like it said in the documentation but I got this error "'khoj' is not recognized as an internal or external command, operable program or batch file."

How can I fix this issue so I can use Khoj?

sabaimran commented 11 months ago

The --user setting would change where the output files are saved to. You might have to update your PATH to include the packages in your user directory.

See this stack overflow post for reference.

How did you install python? There might be something up with your settings which is causing the access denied error. The best way to install is to use the setup executables: https://www.python.org/downloads/.

flowingblaze commented 11 months ago

I thought i installed python using the setup executables at python.org. I went to pythons download page and downloaded python again and it said it was successful so I ran py -m pip install khoj-assistant again. I got an error when trying to install the dependency "pyproject.toml" so I rain pip install pyproject.toml. Both errors are below.

new error when running py -m pip install khoj-assistant with stuff like my username on windows redacted to save my privacy:

Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [96 lines of output]
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: ### Starting.
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: __name__: 'setup'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: platform.platform(): 'Windows-10-10.0.19045-SP0'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: platform.python_version(): '3.12.0'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: sys.executable: 'C:\\Users\\redacted\\AppData\\Local\\Programs\\Python\\Python312\\python.exe'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: CPU bits: 64 sys.maxsize=9223372036854775807
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: __file__: 'C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-install-ok14689n\\pymupdf_6ef4206d02b5422f845c010b5ba245a4\\setup.py'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: os.getcwd(): 'C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-install-ok14689n\\pymupdf_6ef4206d02b5422f845c010b5ba245a4'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: sys.argv (3):
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     0: 'C:\\Users\\redacted\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\pip\\_vendor\\pyproject_hooks\\_in_process\\_in_process.py'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     1: 'prepare_metadata_for_build_wheel'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     2: 'C:\\Users\\redacted\\AppData\\Local\\Temp\\tmplc3ra443'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: os.environ (51):
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     ALLUSERSPROFILE: 'C:\\ProgramData'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     APPDATA: 'C:\\Users\\redacted\\AppData\\Roaming'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     CHOCOLATEYINSTALL: 'C:\\ProgramData\\chocolatey'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     CHOCOLATEYLASTPATHUPDATE: '133401502704798208'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     COMMONPROGRAMFILES: 'C:\\Program Files\\Common Files'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     COMMONPROGRAMFILES(X86): 'C:\\Program Files (x86)\\Common Files'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     COMMONPROGRAMW6432: 'C:\\Program Files\\Common Files'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     COMPUTERNAME: 'redacted'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     COMSPEC: 'C:\\windows\\system32\\cmd.exe'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     DRIVERDATA: 'C:\\Windows\\System32\\Drivers\\DriverData'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     HOMEDRIVE: 'C:'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     HOMEPATH: '\\Users\\redacted'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     JD2_HOME: 'C:\\Users\\redacted\\AppData\\Local\\JDownloader 2.0'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     LOCALAPPDATA: 'C:\\Users\\redacted\\AppData\\Local'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     LOGONSERVER: '\\\\redacted'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     NUMBER_OF_PROCESSORS: '8'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     ONEDRIVE: 'C:\\Users\\redacted\\OneDrive'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     ONLINESERVICES: 'Online Services'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     OS: 'Windows_NT'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PATH: 'C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-build-env-8pk40kz5\\overlay\\Scripts;C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-build-env-8pk40kz5\\normal\\Scripts;C:\\Python311\\Scripts\\;C:\\Python311\\;C:\\Program Files\\Common Files\\Oracle\\Java\\javapath;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\windows\\system32;C:\\windows;C:\\windows\\System32\\Wbem;C:\\windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\windows\\System32\\OpenSSH\\;C:\\Program Files (x86)\\Windows Live\\Shared;C:\\Program Files\\dotnet\\;C:\\Program Files\\Calibre2\\;C:\\Program Files\\Git\\cmd;C:\\Program Files\\nodejs\\;C:\\ProgramData\\chocolatey\\bin;C:\\Users\\redacted\\AppData\\Local\\Programs\\Python\\Python312\\Scripts\\;C:\\Users\\redacted\\AppData\\Local\\Programs\\Python\\Python312\\;C:\\Users\\redacted\\AppData\\Local\\Programs\\Python\\Python310\\Scripts\\;C:\\Users\\redacted\\AppData\\Local\\Programs\\Python\\Python310\\;C:\\Users\\redacted\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\redacted\\AppData\\Local\\GitHubDesktop\\bin;C:\\Users\\redacted\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;c:\\users\\redacted\\.local\\bin;C:\\Users\\redacted\\AppData\\Local\\spicetify;C:\\Users\\redacted\\AppData\\Roaming\\npm;C:\\Users\\redacted\\AppData\\Local\\Programs\\Espanso'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PATHEXT: '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PY;.PYW'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PEP517_BACKEND_PATH: 'C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-install-ok14689n\\pymupdf_6ef4206d02b5422f845c010b5ba245a4'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PEP517_BUILD_BACKEND: 'setup'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PIP_BUILD_TRACKER: 'C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-build-tracker-ffhvnrp9'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PLATFORMCODE: 'KV'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROCESSOR_ARCHITECTURE: 'AMD64'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROCESSOR_IDENTIFIER: 'Intel64 Family 6 Model 126 Stepping 5, GenuineIntel'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROCESSOR_LEVEL: '6'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROCESSOR_REVISION: '7e05'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROGRAMDATA: 'C:\\ProgramData'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROGRAMFILES: 'C:\\Program Files'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROGRAMFILES(X86): 'C:\\Program Files (x86)'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROGRAMW6432: 'C:\\Program Files'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PROMPT: '$P$G'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PSMODULEPATH: 'C:\\Program Files\\WindowsPowerShell\\Modules;C:\\windows\\system32\\WindowsPowerShell\\v1.0\\Modules'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PUBLIC: 'C:\\Users\\Public'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PYTHONNOUSERSITE: '1'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     PYTHONPATH: 'C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-build-env-8pk40kz5\\site'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     REGIONCODE: 'NA'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     SESSIONNAME: 'Console'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     SYSTEMDRIVE: 'C:'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     SYSTEMROOT: 'C:\\windows'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     TEMP: 'C:\\Users\\redacted\\AppData\\Local\\Temp'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     TMP: 'C:\\Users\\redacted\\AppData\\Local\\Temp'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     USERDOMAIN: 'redacted'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     USERDOMAIN_ROAMINGPROFILE: 'redacted'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     USERNAME: 'redacted'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     USERPROFILE: 'C:\\Users\\redacted'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     VBOX_MSI_INSTALL_PATH: 'C:\\Program Files\\Oracle\\VirtualBox\\'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     WINDIR: 'C:\\windows'
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py:     ZES_ENABLE_SYSMAN: '1'
      pipcl.py: build_wheel():  wheel_directory='C:\\Users\\redacted\\AppData\\Local\\Temp\\pip-modern-metadata-kqfpgbbh' config_settings=None metadata_directory=None
      pipcl.py: _call_fn_build(): calling self.fn_build=<function build at 0x000001ED3A37DDA0>
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: skeleton=None
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: PYMUPDF_SETUP_MUPDF_BUILD=None
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: mupdf_tgz already exists: C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\mupdf.tgz
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: Extracting C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\mupdf.tgz
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: Copying C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4/fitz/_config.h to C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\mupdf-1.23.2-source/include/mupdf/fitz/config.h.
      pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py: C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\mupdf-1.23.2-source/include/mupdf/fitz/config.h: os.stat_result(st_mode=33206, st_ino=281474978733499, st_dev=17783704003988180549, st_nlink=1, st_uid=0, st_gid=0, st_size=5686, st_atime=1696269149, st_mtime=1695760732, st_ctime=1696269128) mtime=2023-09-26-20:38:52
      Traceback (most recent call last):
        File "C:\Users\redacted\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\redacted\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\redacted\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\pipcl.py", line 582, in build_wheel
          items = self._call_fn_build(config_settings)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\pipcl.py", line 731, in _call_fn_build
          ret = self.fn_build()
                ^^^^^^^^^^^^^^^
        File "C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py", line 672, in build
          mupdf_build_dir = build_mupdf_windows( mupdf_local, env_extra, build_type)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\setup.py", line 802, in build_mupdf_windows
          vs = pipcl.wdev.WindowsVS()
               ^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\redacted\AppData\Local\Temp\pip-install-ok14689n\pymupdf_6ef4206d02b5422f845c010b5ba245a4\wdev.py", line 81, in __init__
          assert os.path.isfile( devenv), f'Does not exist: {devenv}'
      AssertionError: Does not exist: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\devenv.com
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

C:\Users\redacted>

and the error I got with pip install pyproject.toml:

 WARNING: Failed to write executable - trying to use .deleteme logic
ERROR: Could not install packages due to an OSError: [WinError 2] The system cannot find the file specified: 'C:\\Python311\\Scripts\\wheel.exe' -> 'C:\\Python311\\Scripts\\wheel.exe.deleteme'

sabaimran commented 11 months ago

Ah, based on this log:

AssertionError: Does not exist: C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\devenv.com [end of output]

It looks like there might be an issue with the c++ dependencies. Can you reference the prerequisites portion of this documentation and see if that resolved the error? You'll have to install a couple of files to get it up and working.

flowingblaze commented 11 months ago

Didn't work. I got what looks like to be the same error.

debanjum commented 10 months ago

Hey @flowingblaze, Can you try install wheel with pip install wheel before trying to install khoj with pip install khoj-assistant? The issue may just be that wheel is a dependency of khoj but we're not listing it as such

flowingblaze commented 10 months ago

"Requirement already satisfied: wheel in c:\python311\lib\site-packages (0.41.2)"

putsan commented 10 months ago

Today I'll write briefly (as I've already wasted several hours), I couldn't get Khoj Backend running on either Win 10 or Linux Mint 20.03.

I tried everything possible to make it work, and even GPT couldn't help me. When I try again, I think I'll write in the Discord chat and maybe on a GitHub Issue.

This is because on both operating systems, I was hindered by numerous dependency incompatibilities, which I haven't been able to fix yet.

debanjum commented 10 months ago

Hey @putsan, @flowingblaze, we've released a new version of Khoj that has a cleaner client, server separation. This should simplify setting up the Khoj Obsidian plugin:

You'd just need to generate a token on your Khoj cloud account
Set it in the Khoj Obsidian plugin settings to search, chat with your Vault

You can also reference the docs for these new setup instructions.

To self-host the Khoj server you can use Docker to set it up on your machine by following the self-host docs. This setup step is optional, as it requires a bit of technical proficiency (knowledge of docker, terminal etc.)

Let us know if you still aren't able to use Khoj from Obsidian on Windows using these new setup steps?

flowingblaze commented 10 months ago

I cannot make a Khoj cloud account. The only option I see is to sign in with google and I try to do that I get an error saying "an error has occured. That's all we know".

sabaimran commented 10 months ago

@flowingblaze , could you send me a screenshot of that? Would you be able to try with another browser? Looks like the issue is on Google's side, but I don't see a place for any aggregated logs.

sabaimran commented 10 months ago

@putsan , please do feel free to open a new issue and document the roadblocks you're hitting. We're working actively on addressing any developer setup problems. It's probably best to create a new issue as this one has drifted from the original problem.

debanjum commented 7 months ago

Hey folks, I'm closing this issue as it's become stale/unactionable with the general improvements and rearchitecture of Khoj cloud over the last few months. But do please feel free to open separate, specific issues if you're still unable to use Khoj (self-hosted or cloud).

khoj-ai / khoj

Can't install Khoj Backend #490