installation for windows

[NewGraphEnvironment]

Incredible tool. Hope to get it running. Working on installing this to windows. So far i have done the following. I am not experienced with most of these tools so please pardon my ignorance and newbieisms.

I did not:

• get started with a fresh Windows Subsystem for Linux (WSL) (Ubuntu 20.04.1 LTS) - maybe this was my first mistake?!

I did:

• Install virt env and bcdata as per the instructions at bcfishpass
• Download wget (actually just found it elsewhere on my system) and install into the same bcfp virt env directory
• git clone the bcfishobs repo locally https://github.com/smnorris/bcfishobs.git
• rename 01_load.sh to 01_load.bat and 02_process.sh to 02_process.bat
• change line endings in both bat files from \ to ^

When trying to run 01_load.bat from the Anaconda prompt at cd bcfishobs and within the bcfp virt env issue arose at:

1. (bcfp) C:\scripts\bcfishobs>ogr2ogr -f PostgreSQL "PG:host=$PGHOST user=$PGUSER dbname=$PGDATABASE port=$PGPORT" -lco OVERWRITE=YES -lco SCHEMA=whse_fish -nln wdic_waterbodies_load -nlt NONE whse_fish.wdic_waterbodies.csv

fails with

ERROR 1: PQconnectdb failed.
invalid integer value "$PGPORT" for connection option "port"

tried

export PGOGR='host=localhost user=postgres dbname=postgres password=postgres port=5432'

and

export PGHOST=localhost etc.

as per README.md on bcfishpass to no prevail with message

'export' is not recognized as an internal or external command

below is a screen shot of my env variables. Should they have dollar signs in front of them perhaps?

2. line 45 of 01_load.bat fails psql -c "CREATE TABLE whse_fish.fiss_obstacles_unpublished (id integer ..........due to unexpected line endings. Could not seem to find a solution other than re-writing the command with no spaces and copying into the terminal directly. This worked but is there another way that you know of?

3. bcdata bc2pg WHSE_FISH.FISS_OBSTACLES_PNT_SP --fid FISH_OBSTACLE_POINT_ID fails with the following error message: Any idea what might be going wrong? Thank you!

(bcfp) C:\scripts\bcfishobs>bcdata bc2pg WHSE_FISH.FISS_OBSTACLES_PNT_SP --fid FISH_OBSTACLE_POINT_ID
Traceback (most recent call last):
  File "c:\users\allan\.conda\envs\bcfp\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\allan\.conda\envs\bcfp\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\allan\.conda\envs\bcfp\Scripts\bcdata.exe\__main__.py", line 7, in <module>
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\bcdata\cli.py", line 350, in bc2pg
    conn = pgdata.connect(db_url)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\pgdata\__init__.py", line 20, in connect
    return Database(url, schema, sql_path=sql_path, multiprocessing=multiprocessing)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\pgdata\database.py", line 41, in __init__
    self.engine = create_engine(url)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\sqlalchemy\engine\__init__.py", line 500, in create_engine
    return strategy.create(*args, **kwargs)
  File "c:\users\allan\.conda\envs\bcfp\lib\site-packages\sqlalchemy\engine\strategies.py", line 56, in create
    plugins = u._instantiate_plugins(kwargs)
AttributeError: 'NoneType' object has no attribute '_instantiate_plugins'

[smnorris]

Definitely try WSL, this should help a lot.

1. Windows command for setting environment is SET, not export. https://superuser.com/questions/79612/setting-and-getting-windows-environment-variables-from-the-command-prompt

2. First step would be try converting the line endings from unix to windows but I'm not sure

3. Not sure but probably the issue is that your environment variables aren't set (although I'm not sure how to evaluate your screen shot). Try specifying the db directly with the db_url option: bcdata bc2pg bc-airports --db_url postgresql://postgres:postgres@localhost:5432/postgis

[NewGraphEnvironment]

still planning on looking into this. thanks for keeping the issue open for now

[NewGraphEnvironment]

Another 24.75hrs later. no luck. 🤮

[NewGraphEnvironment]

not your problem. bcdata bc2pg bc-airports --db_url postgresql://postgres:postgres@localhost:5432/postgis actually works on my windows command line now but couldn't set env variables there. Installed wsl and separate pg instance but cant connect through bcdata. So insanely difficult for me to figure it out for some reason. I feel like such a loser.

[smnorris]

If you create & share a shell script I can point to any issues I can spot.

[smnorris]

This will set up the env variables in windows command prompt.

SET PGHOST=localhost
SET PGUSER=posgres
SET PGDATABASE=mydb 
SET PGPORT=5432
SET DATABASE_URL=postgresql://%PGUSER%:mypassword@%PGHOST%:%PGPORT%/%PGDATABASE%

[NewGraphEnvironment]

sent a invite to a repo i made to try to keep track of what was going on. pg_wsl_setup.Rmd has some of the outputs I get when trying to connect in wsl. Might be something that jumps out at you as incompetent but I doubt it. Its going to be hard to follow

[smnorris]

@NewGraphEnvironment - I think you got this sorted?

[NewGraphEnvironment]

Yes. I can’t quite remember what went on but using wsl2 on window and the virt env that ships with bcfishpass worked