DocSearch Scraper fails to start with Docker

[jbremmer]

Description

Running Docker as instructed here https://typesense.org/docs/guide/docsearch.html#run-the-scraper

fails with Pipenv error.

Steps to reproduce

run latest typesense/docsearch-scraper in Docker

Expected Behavior

DocSarch Scraper starting as expected.

Actual Behavior

Startup craches with

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/dist-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/dist-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/cli/command.py", line 450, in run
    command=command, args=args, three=state.three, python=state.python, pypi_mirror=state.pypi_mirror
  File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 2513, in do_run
    three=three, python=python, validate=False, pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 580, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 493, in ensure_virtualenv
    if not project.virtualenv_exists:
  File "/usr/local/lib/python3.6/dist-packages/pipenv/project.py", line 271, in virtualenv_exists
    if os.path.exists(self.virtualenv_location):
  File "/usr/local/lib/python3.6/dist-packages/pipenv/project.py", line 470, in virtualenv_location
    self._virtualenv_location = self.get_location_for_virtualenv()
  File "/usr/local/lib/python3.6/dist-packages/pipenv/project.py", line 294, in get_location_for_virtualenv
    return str(get_workon_home().joinpath(self.virtualenv_name))
  File "/usr/local/lib/python3.6/dist-packages/pipenv/utils.py", line 2016, in get_workon_home
    mkdir_p(str(expanded_path))
  File "/usr/local/lib/python3.6/dist-packages/pipenv/utils.py", line 1488, in mkdir_p
    mkdir_p(head)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/utils.py", line 1488, in mkdir_p
    mkdir_p(head)
  File "/usr/local/lib/python3.6/dist-packages/pipenv/utils.py", line 1493, in mkdir_p
    os.mkdir(newdir)
PermissionError: [Errno 13] Permission denied: '/.local'

Metadata

- Typsense Version: N/A

OS:

N/A - Docker

[augustluhrs]

can you post the command you're running? it seems like it might just be a typo in the path (putting the period after the slash)

[jbremmer]

This was actually on Kubernetes, here's how I provide it to container:

env:
  - name: TYPESENSE_API_KEY
    value: 2b...cbf
  - name: TYPESENSE_HOST
    value: <host>
  - name: TYPESENSE_PORT
    value: '80'
  - name: TYPESENSE_PROTOCOL
    value: http
  - name: CONFIG
    value: >-
      {"index_name":"typesense_docs","start_urls":[{"url":"https://typesense.org/docs/(?P<version>.*?)/","variables":{"version":["0.22.2","0.22.1","0.22.0","0.21.0","0.20.0","0.19.0","0.18.0","0.17.0","0.16.1","0.16.0","0.15.0","0.14.0","0.13.0","0.12.0","0.11.2"]}}],"selectors":{"default":{"lvl0":".content__default h1","lvl1":".content__default h2","lvl2":".content__default h3","lvl3":".content__default h4","lvl4":".content__default h5","text":".content__default p, .content__default ul li, .content__default table tbodytr"}},"scrape_start_urls":false,"strip_chars":" .,;:#"}