search

linkml / archived-linkml-model-template

DEPRECATED! Please use linkml-project-cookiecutter instead.
10 stars 6 forks source link

Bug/UX Issue: `FileNotFoundError` for `types.yaml` (unaltered `sample_model.yaml` causes errors during `make` setup) #12

Closed joeflack4 closed 1 year ago

joeflack4 commented 3 years ago

Description

When setting up a linkml project using the template setup and making no alterations to the sample_model.yaml file, a FileNotFoundError (types.yaml) will appear.

Error message

FileNotFoundError: [Errno 2] No such file or directory: '/Users/joeflack4/projects/linkml-test-template/model/schema/types.yaml'

Steps to reproduce

  1. Use this template
  2. Clone repo
  3. Use the attached config file
  4. Run the make commands

More info

Logs

Joes-2015-MacBook:linkml-test-template joeflack4$ make -f MakeConfig reset
. venv/bin/activate && configure model/CONFIG.yaml --reset
mkdocs.yml written
requirements.txt written
Makefile written
README.md written
Pipfile written
setup.py written
.gitignore written
tox.ini written
setup.cfg written
tests/test_input_against_model.py written
tests/__init__.py written
tests/input/CONFIG.yaml written
tests/input/README.md written
make-venv/README.md written
make-venv/Pipfile written
.github/workflows/pypi-publish.yaml written
.github/workflows/main.yaml written
model/docs/credits.md written
model/docs/home.md written
model/schema/README.md written
Joes-2015-MacBook:linkml-test-template joeflack4$ make
rm -rf target/docs
mkdir -p target/docs
export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv install
Creating a virtualenv for this project...
Pipfile: /Users/joeflack4/projects/linkml-test-template/make-venv/Pipfile
Using /Library/Frameworks/Python.framework/Versions/3.9/bin/python3 (3.9.6) to create virtualenv...
⠦ Creating virtual environment...created virtual environment CPython3.9.6.final.0-64 in 924ms
  creator CPython3Posix(dest=/Users/joeflack4/projects/linkml-test-template/make-venv/.venv, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/joeflack4/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment!
Virtualenv location: /Users/joeflack4/projects/linkml-test-template/make-venv/.venv
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (449c22)!
Installing dependencies from Pipfile.lock (449c22)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 57/57 — 00:01:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
touch make-venv/env.lock
export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-markdown --log_level WARNING --mergeimports --notypesdir --warnonexist --dir target/docs model/schema/sample_model.yaml
Traceback (most recent call last):
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/bin/gen-markdown", line 8, in <module>
    sys.exit(cli())
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/generators/markdowngen.py", line 579, in cli
    MarkdownGenerator(yamlfile, no_types_dir=notypesdir, warn_on_exist=warnonexist, **kwargs)\
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/generators/markdowngen.py", line 25, in __init__
    super().__init__(schema, **kwargs)
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/utils/generator.py", line 92, in __init__
    loader.resolve()
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/utils/schemaloader.py", line 101, in resolve
    load_raw_schema(sname + '.yaml',
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/utils/rawloader.py", line 64, in load_raw_schema
    schema = yaml_loader.load(copy.deepcopy(data) if isinstance(data, dict) else data,
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml_runtime/loaders/yaml_loader.py", line 22, in load
    return self.load_source(source, loader, target_class, accept_header="text/yaml, application/yaml;q=0.9",
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml_runtime/loaders/loader_root.py", line 56, in load_source
    data = hbread(source, metadata, metadata.base_path, accept_header)
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/hbreader/__init__.py", line 260, in hbread
    with hbopen(source, open_info, base_path, accept_header, is_actual_data, read_codec) as f:
  File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/hbreader/__init__.py", line 206, in hbopen
    f = open(fname, encoding=read_codec if read_codec else 'utf-8')
FileNotFoundError: [Errno 2] No such file or directory: '/Users/joeflack4/projects/linkml-test-template/model/schema/types.yaml'
make: *** [target/docs/index.md] Error 1

Other things I tried

I tried using another example from the linkml docs which has a types field:

id: http://example.org/sample/organization
name: organization

types:
  yearCount:
    base: int
    uri: xsd:int
  string:
    base: str
    uri: xsd:string

classes:

  organization:
    slots:
      - id
      - name
      - has boss

  employee:
    description: A person
    slots:
      - id
      - first name
      - last name
      - aliases
      - age in years
    slot_usage:
      last name :
        required: true

  manager:
    description: An employee who manages others
    is_a: employee
    slots:
      - has employees

slots:
  id:
    description: Unique identifier of a person
    identifier: true

  name:
    description: human readable name
    range: string

  aliases:
    is_a: name
    description: An alternative name
    multivalued: true

  first name:
    is_a: name
    description: The first name of a person

  last name:
    is_a: name
    description: The last name of a person

  age in years:
    description: The age of a person if living or age of death if not
    range: yearCount
[CONFIG.yaml.zip](https://github.com/linkml/linkml-model-template/files/6729877/CONFIG.yaml.zip)

  has employees:
    range: employee
    multivalued: true
    inlined: true

  has boss:
    range: manager
    inlined: true

Dump of everything I tried from my note taking app "Workflowy":

Fix issues

- types.yaml error
  - err msg
    - FileNotFoundError: [Errno 2] No such file or directory: '/Users/joeflack4/projects/linkml-test-template/model/schema/types.yaml'
  - solutions?
    - a. Try a different schema (X)
      - Don't remember where I go tit, but I used an example from one of the slides / documentation.
      - err
        - (don't remember error, but didn't work)
    - b. fix typo? (X)
      - the typo
        - type --> linkml:type
        - I found this here in these slides
          - https://docs.google.com/presentation/d/1jtV2vQhCwsGv_9fPplkG8ww1ru4V_JYyRGLAm2dxvdE/edit#slide=id.gced59d45c7_0_0
      - result: err
        - err
          - err short text
            - make: *** No rule to make target `gen-jsonld', needed by `gen-rdf'.  Stop.
          - err full text
            - make
            - rm -rf target/docs
            - mkdir -p target/docs
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv install
            - Installing dependencies from Pipfile.lock (449c22)...
              - 🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
            - To activate this project's virtualenv, run pipenv shell.
            - Alternatively, run a command inside the virtualenv with pipenv run.
            - touch make-venv/env.lock
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-markdown --log_level WARNING --mergeimports --notypesdir --warnonexist --dir target/docs model/schema/sample_model.yaml
            - mkdir -p model/docs
            - cp -R model/docs/*.md target/docs
            - # mkdocs.yml moves from the target/docs to the docs directory
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run mkdocs build
            - WARNING  -  Config value: 'site_url'. Warning: This option is now required. Set to a valid URL or an empty string to avoid an
              - error in a future release.
            - WARNING  -  Config value: 'site_url'. Warning: The 'use_directory_urls' option has been disabled because 'site_url' contains an
              - empty value. Either define a valid URL for 'site_url' or set 'use_directory_urls' to False.
            - INFO     -  Cleaning site directory
            - INFO     -  Building documentation to directory: /Users/joeflack4/projects/linkml-test-template/docs
            - INFO     -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
              - Boolean.md
                - Date.md
                  - Datetime.md
                  - Decimal.md
                  - Double.md
                  - Float.md
                  - FriendlyPerson.md
                  - FriendlyPerson_knows.md
                  - Integer.md
                  - Ncname.md
                  - Nodeidentifier.md
                  - Objectidentifier.md
                  - Person.md
                  - String.md
                  - Time.md
                  - Uri.md
                  - Uriorcurie.md
                  - person__first_name.md
                  - person__id.md
                  - person__knows.md
                  - person__last_name.md
            - INFO     -  Documentation built in 0.38 seconds
            - rm -rf target/jsonld_context
            - mkdir -p target/jsonld_context
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-jsonld-context --log_level WARNING --no-mergeimports model/schema/sample_model.yaml > target/jsonld_context/sample_model.context.jsonld
            - mkdir -p linkml_test_template/jsonld
            - cp target/jsonld_context/sample_model.context.jsonld linkml_test_template/jsonld/sample_model.context.jsonld
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-jsonld-context --log_level WARNING --no-mergeimports model/schema/sample_model.yaml > target/jsonld_context/sample_model.model.context.jsonld
            - mkdir -p linkml_test_template/jsonld
            - cp target/jsonld_context/sample_model.model.context.jsonld linkml_test_template/jsonld/sample_model.model.context.jsonld
            - rm -rf target/json_schema
            - mkdir -p target/json_schema
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-json-schema --log_level WARNING -t transaction model/schema/sample_model.yaml > target/json_schema/sample_model.schema.json
            - mkdir -p linkml_test_template/jsonschema
            - cp target/json_schema/sample_model.schema.json linkml_test_template/jsonschema/sample_model.schema.json
            - rm -rf target/graphql
            - mkdir -p target/graphql
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-graphql --log_level WARNING model/schema/sample_model.yaml > target/graphql/sample_model.graphql
            - mkdir -p linkml_test_template/graphql
            - cp target/graphql/sample_model.graphql linkml_test_template/graphql/sample_model.graphql
            - rm -rf target/owl
            - mkdir -p target/owl
            - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-owl --log_level WARNING model/schema/sample_model.yaml > target/owl/sample_model.owl.ttl
            - mkdir -p linkml_test_template/owl
            - cp target/owl/sample_model.owl.ttl linkml_test_template/owl/sample_model.owl.ttl
            - make: *** No rule to make target `gen-jsonld', needed by `gen-rdf'.  Stop.
        - solutions?
          - 1. target: gen-json --> gen-jsonld
            - rationale
              - gen-jsonld was referred to but not defined. gen-json was defined but not referred to. supposed to be one and the same?
            - result?
              - err: SSL
                - err short text
                  - urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)>
                - err long text
                  - make
                  - mkdir -p model/docs
                  - cp -R model/docs/*.md target/docs
                  - # mkdocs.yml moves from the target/docs to the docs directory
                  - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run mkdocs build
                  - WARNING  -  Config value: 'site_url'. Warning: This option is now required. Set to a valid URL or an empty string to avoid an
                    - error in a future release.
                  - WARNING  -  Config value: 'site_url'. Warning: The 'use_directory_urls' option has been disabled because 'site_url' contains an
                    - empty value. Either define a valid URL for 'site_url' or set 'use_directory_urls' to False.
                  - INFO     -  Cleaning site directory
                  - INFO     -  Building documentation to directory: /Users/joeflack4/projects/linkml-test-template/docs
                  - INFO     -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
                    - Boolean.md
                  - INFO     -  Documentation built in 0.98 seconds
                  - rm -rf target/json
                  - mkdir -p target/json
                  - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-jsonld --log_level WARNING --no-mergeimports model/schema/sample_model.yaml > target/json/sample_model.json
                  - mkdir -p linkml_test_template/json
                  - cp target/json/sample_model.json linkml_test_template/json/sample_model.json
                  - rm -rf target/rdf
                  - mkdir -p target/rdf
                  - export PIPENV_VENV_IN_PROJECT=true && export PIPENV_PIPFILE=make-venv/Pipfile && export PIPENV_IGNORE_VIRTUALENVS=1 && pipenv run gen-rdf --log_level WARNING --context /Users/joeflack4/projects/linkml-test-template/linkml_test_template/jsonld/sample_model.context.jsonld model/schema/sample_model.yaml > target/rdf/sample_model.ttl
                  - Traceback (most recent call last):
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1346, in do_open
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1257, in request
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1303, in _send_request
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1252, in endheaders
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1012, in _send_output
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 952, in send
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1426, in connect
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake
                  - ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
                  - During handling of the above exception, another exception occurred:
                  - Traceback (most recent call last):
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/bin/gen-rdf", line 8, in <module>
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 829, in __call__
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 782, in main
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/generators/rdfgen.py", line 58, in cli
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/utils/generator.py", line 132, in serialize
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/linkml/generators/rdfgen.py", line 43, in end_schema
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib/graph.py", line 1078, in parse
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib_jsonld/parser.py", line 95, in parse
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib_jsonld/parser.py", line 107, in to_rdf
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib_jsonld/parser.py", line 125, in parse
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib_jsonld/context.py", line 200, in load
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib_jsonld/context.py", line 213, in _prep_sources
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib_jsonld/util.py", line 23, in source_to_json
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib/parser.py", line 193, in create_input_source
                    - File "/Users/joeflack4/projects/linkml-test-template/make-venv/.venv/lib/python3.9/site-packages/rdflib/parser.py", line 113, in __init__
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1389, in https_open
                    - File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1349, in do_open
                  - urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)>
                  - make: *** [target/rdf/sample_model.ttl] Error 1
              - solutions?
                - tried (X)
                  - https://stackoverflow.com/questions/50236117/scraping-ssl-certificate-verify-failed-error-for-http-en-wikipedia-org
                - https://stackoverflow.com/questions/44649449/brew-installation-of-python-3-6-1-ssl-certificate-verify-failed-certificate/44649450#44649450

Possible solutions

Old ideas

It looks like this is happening because the sample_model.yaml is not a "well formed" LinkML spec file, so to speak. It looks like it is missing some possibly required keys, namely "slots" and "types". Additionally, it has a line imports: - types. I'm not sure if this (a) is something that is related but not causing the error, (b) something that's directly helping cause the error, or (c) is something that is supposed to help prevent the error (e.g. it imports some basic types?) but is not working.

linkml:types solution

Came up w/ the idea by seeing it used here: https://docs.google.com/presentation/d/1jtV2vQhCwsGv_9fPplkG8ww1ru4V_JYyRGLAm2dxvdE/edit#slide=id.gcf33f470c2_1_138

Screen Shot 2021-07-03 at 9 49 21 PM
joeflack4 commented 3 years ago

On the usability side of things, I think it would be nice to have a sample_model.yaml that the user does not need to modify for the purposes of the quick start.

joeflack4 commented 3 years ago

Fixes

Should be fixed by merging the following PRs: https://github.com/linkml/template-configurator/pull/14 https://github.com/linkml/linkml-model-template/pull/14

joeflack4 commented 3 years ago

14 has been merged, so this can probably be closed now, unless someone else wants to verify.