mksetup: resulting setup.py not working

xnopasaranx commented 6 years ago

tried to clone and setup a fresh base project, but following the steps described in the readme leaves me with a broken project. It seems like the clean-base script is using different assumptions about the project structure which ultimately messes everything up.

After cloning the repo, I ran "python2 mksetup.py > setup.py", then executed the resulting setup.py, the readme then suggests running clean-base.py to clean up. After running clean-base.py the setup is incomplete and the application cannot be run with local_run.sh, as the directory structure is not as expected. Trying again and not running clean-base.py also results in a broken project.

Cloning the repo and only running clean-base.py results in a working project which can be started through local_run.sh

xnopasaranx commented 6 years ago

Tried to debug this and cannot figure out exactly what setup.py does regarding directory structure. There is a number of things that are done redundantly and which clean-base subsequently clobbers.

The script does not differentiate between python2 and python3 installs, so running it simply with python mksetup.py > setup.py will fail on systems which default to python3
it downloads vi and server from the latest builds url, instead of using git like clean-base.py is doing, which leads to unpleasantness.

We should throw out at least one of these scripts! There needs to be a consistent way to initialize a fresh project.

phorward commented 6 years ago

Hello @xnopasaranx,

The setup.py script is generated from the base project's deploy directory's contents to provide a way to quickly setup a ViUR project with the latest server and vi, without any compromise. It is the simplest and easiest way to start with a fresh ViUR project. There is no other dependency than just gcloud SDK to run it.
The clean-base.py script uses the base project's git repository itself as a development base for a new project, which is intended to be managed by git and uses all required ViUR modules as submodules to be easier updated. This way is the "professional" way in how to set up a clean base project for a new application that will be developed on top of it, including customized Vi extensions and third-party libraries.

I tested both way, running the generated setup.py that fell our from the same call as you generated it, and running clean-base.py after cloning base into a fresh directory, both ways work well and as expected, so there might be a problem with your Python versions or another issue that causes your tests to fail.

Here is my protocol of the setup.py-based way:

neo@falcon ~/Mausbrand/base $ mkdir /tmp/v
neo@falcon ~/Mausbrand/base $ python2 mksetup.py >/tmp/v/setup.py
neo@falcon ~/Mausbrand/base $ cd /tmp/v
neo@falcon /tmp/v $ python2 setup.py

                iii
               iii
              iii

          vvv iii uu      uu rrrrrrrr
         vvvv iii uu      uu rr     rr
  v     vvvv  iii uu      uu rr     rr
 vvv   vvvv   iii uu      uu rr rrrrr
vvvvv vvvv    iii uu      uu rr rrr
 vvvvvvvv     iii uu      uu rr  rrr
  vvvvvv      iii  uu    uu  rr   rrr
   vvvv       iii   uuuuuu   rr    rrr

  I N F O R M A T I O N    S Y S T E M

Welcome to the raw setup utility!

Please enter your desired application name [default=v]
This will setup a clean ViUR project 'v' now - continue [Y/n]?
Creating /tmp/v/emails...
Creating /tmp/v/html...
Creating /tmp/v/lib...
Creating /tmp/v/modules...
Creating /tmp/v/skeletons...
Creating /tmp/v/static...
Creating /tmp/v/translations...
Creating /tmp/v/html/icons...
Creating /tmp/v/html/macros...
Creating /tmp/v/html/sites...
Creating /tmp/v/static/css...
Creating /tmp/v/static/images...
Creating /tmp/v/static/js...
Creating /tmp/v/static/less...
Creating /tmp/v/static/meta...
Creating /tmp/v/static/webfonts...
Writing html/person_view.html...Done
Writing static/meta/viur-apple-touch-icon-72x72-precomposed.png...Done
Writing html/icons/not-allowed.svg...Done
Writing html/macros/skelview.html...Done
Writing html/icons/arrowhead-left.svg...Done
Writing skeletons/person.py...Done
Writing html/icons/check.svg...Done
Writing static/js/jquery-3.2.1.min.js...Done
Writing static/meta/viur-apple-touch-icon-57x57-precomposed.png...Done
Writing html/viur_default_view.html...Done
Writing index.yaml...Done
Writing static/meta/favicon.ico...Done
Writing html/user_login_secondfactor.html...Done
Writing emails/user_password_recovery.email...Done
Writing html/viur_default_list.html...Done
Writing modules/index.py...Done
Writing app.yaml...Done
Writing emails/viur_mail_dummy.email...Done
Writing html/user_passwordrecover.html...Done
Writing static/meta/.keep...Done
Writing appengine_config.py...Done
Writing html/macros/functions.html...Done
Writing static/meta/favicon.png...Done
Writing html/user_login_success.html...Done
Writing static/images/.keep...Done
Writing lib/.keep...Done
Writing static/less/style.less...Done
Writing html/user_logout_success.html...Done
Writing html/user_passwordrecover_already_sent.html...Done
Writing queue.yaml...Done
Writing translations/en.py...Done
Writing static/meta/viur-apple-touch-startup-image-320x460.png...Done
Writing html/icons/heart.svg...Done
Writing html/viur_base.html...Done
Writing html/icons/cross.svg...Done
Writing modules/file.py...Done
Writing static/js/pagination.js...Done
Writing static/images/start-vi.png...Done
Writing html/index.html...Done
Writing html/sites/.keep...Done
Writing static/meta/viur-apple-touch-startup-image-1536x2008.png...Done
Writing static/meta/viur_micro.gif...Done
Writing static/webfonts/.keep...Done
Writing modules/user.py...Done
Writing static/css/style.css...Done
Writing static/js/app.js...Done
Writing static/meta/viur-apple-touch-startup-image-768x1004.png...Done
Writing cron.yaml...Done
Writing translations/__init__.py...Done
Writing html/sites/gettingstarted.html...Done
Writing static/css/style.min.css...Done
Writing modules/person.py...Done
Writing html/user_passwordrecover_invalid_token.html...Done
Writing static/meta/viur-opengraph-image.png...Done
Writing modules/__init__.py...Done
Writing static/meta/viur-logo-error.png...Done
Writing html/viur_mail_default.html...Done
Writing html/person_list.html...Done
Writing skeletons/__init__.py...Done
Writing html/icons/flash.svg...Done
Writing v.py...Done
Writing html/user_passwordrecover_mail_sent.html...Done
Writing static/meta/viur-apple-touch-icon-precomposed.png...Done
Writing static/meta/viur-apple-touch-startup-image-748x1024.png...Done
Writing html/user_passwordrecover_success.html...Done
Writing translations/de.py...Done
Writing html/icons/pen.svg...Done
Writing static/meta/humans.txt...Done
Writing static/meta/viur-apple-touch-icon-144x144-precomposed.png...Done
Writing static/meta/viur-apple-touch-icon-114x114-precomposed.png...Done
Writing static/meta/robots.txt...Done
Writing html/user_login.html...Done
Writing static/meta/mausbrand_micro.gif...Done
Writing static/meta/viur-apple-touch-startup-image-1496x2048.png...Done
Writing html/icons/token.svg...Done
Writing static/meta/viur-apple-touch-startup-image-640x920.png...Done
Downloading server...Done
Extracting server...Done
Downloading vi...Done
Extracting vi...Done
--- ViUR setup completed. ---

For local development server startup, run

        dev_appserver.py -A v .

To deploy your app to Google App Engine, create your project first, then type

        gcloud app deploy -q --project v

Visit https://docs.viur.is for more information.
Please do not forget to remove setup.py now!
neo@falcon /tmp/v $ dev_appserver.py -A v . 
INFO     2018-03-28 12:25:56,604 devappserver2.py:105] Skipping SDK update check.
WARNING  2018-03-28 12:25:56,684 simple_search_stub.py:1196] Could not read search indexes from /tmp/appengine.v.neo/search_indexes
INFO     2018-03-28 12:25:56,685 api_server.py:265] Starting API server at: http://localhost:39539
INFO     2018-03-28 12:25:56,699 dispatcher.py:255] Starting module "default" running at: http://localhost:8080
INFO     2018-03-28 12:25:56,699 admin_server.py:152] Starting admin server at: http://localhost:8000
INFO     2018-03-28 12:25:59,797 module.py:835] default: "GET /_ah/warmup HTTP/1.1" 200 2
WARNING  2018-03-28 12:25:59,998 user.py:694] ViUR created a new admin-user for you! Username: admin@v.appspot.com, Password: FDepGkxk02MfJ
INFO     2018-03-28 12:26:00,000 mail_stub.py:141] MailService.SendToAdmins
  From: viur@v.appspotmail.com
  Subject: =?utf-8?B?WW91ciBuZXcgVmlVUiBwYXNzd29yZA==?=
  Body:
    Content-type: text/plain
    Data length: 93
INFO     2018-03-28 12:26:00,008 module.py:835] default: "POST /_tasks/deferred HTTP/1.1" 200 4
INFO     2018-03-28 12:26:02,801 module.py:835] default: "GET /_ah/warmup HTTP/1.1" 200 2
INFO     2018-03-28 12:26:02,876 module.py:835] default: "POST /_tasks/deferred HTTP/1.1" 200 4
INFO     2018-03-28 12:26:05,203 module.py:835] default: "GET / HTTP/1.1" 200 7938
INFO     2018-03-28 12:26:05,290 module.py:835] default: "GET /static/js/jquery-3.2.1.min.js HTTP/1.1" 304 -
INFO     2018-03-28 12:26:05,298 module.py:835] default: "GET /static/css/ignite.min.css HTTP/1.1" 404 -
INFO     2018-03-28 12:26:05,299 module.py:835] default: "GET /static/js/app.js HTTP/1.1" 304 -
INFO     2018-03-28 12:26:05,300 module.py:835] default: "GET /static/css/style.min.css HTTP/1.1" 304 -
^CINFO     2018-03-28 12:26:12,375 shutdown.py:45] Shutting down.
INFO     2018-03-28 12:26:12,375 stub_util.py:360] Applying all pending transactions and saving the datastore
INFO     2018-03-28 12:26:12,379 stub_util.py:363] Saving search indexes
neo@falcon /tmp/v $

and here is my protocol of the clean-base.py-based way:

neo@falcon ~ $ git clone Mausbrand/base/ /tmp/v2
neo@falcon ~ $ cd /tmp/v2/
neo@falcon /tmp/v2 $ ./clean-base.py 
Enter Author Name (leave empty to default to neo): 
Enter application-id (leave empty to default to v2-viur): 
Downloading submodules
Submodule 'deploy/server' (https://github.com/viur-framework/server.git) registered for path 'deploy/server'
Submodule 'ignite' (https://github.com/viur-framework/ignite.git) registered for path 'ignite'
Submodule 'vi' (https://github.com/viur-framework/vi.git) registered for path 'vi'
Cloning into '/tmp/v2/deploy/server'...
Cloning into '/tmp/v2/ignite'...
Cloning into '/tmp/v2/vi'...
Submodule 'html5' (https://github.com/viur-framework/html5.git) registered for path 'html5'
Submodule 'logics' (https://github.com/viur-framework/logics.git) registered for path 'logics'
Submodule 'public/icons' (https://github.com/viur-framework/icons.git) registered for path 'public/icons'
Cloning into '/tmp/v2/vi/html5'...
Cloning into '/tmp/v2/vi/logics'...
Cloning into '/tmp/v2/vi/public/icons'...
Removing .git tether
.git remote tether removed
Downloading latest build of vi...Done downloading latest build of vi
Extracting latest build of vi...Done extracting vi
neo@falcon /tmp/v2 $ dev_appserver.py -A viur-v2 deploy/
INFO     2018-03-28 12:22:52,224 devappserver2.py:105] Skipping SDK update check.
INFO     2018-03-28 12:22:52,301 api_server.py:265] Starting API server at: http://localhost:34783
INFO     2018-03-28 12:22:52,315 dispatcher.py:255] Starting module "default" running at: http://localhost:8080
INFO     2018-03-28 12:22:52,316 admin_server.py:152] Starting admin server at: http://localhost:8000
INFO     2018-03-28 12:22:55,391 module.py:835] default: "GET /_ah/warmup HTTP/1.1" 200 2
WARNING  2018-03-28 12:22:55,607 user.py:694] ViUR created a new admin-user for you! Username: admin@viur-v2.appspot.com, Password: UxQK8z7ZuD2zc
INFO     2018-03-28 12:22:55,609 mail_stub.py:141] MailService.SendToAdmins
  From: viur@viur-v2.appspotmail.com
  Subject: =?utf-8?B?WW91ciBuZXcgVmlVUiBwYXNzd29yZA==?=
  Body:
    Content-type: text/plain
    Data length: 99
INFO     2018-03-28 12:22:55,615 module.py:835] default: "POST /_tasks/deferred HTTP/1.1" 200 4
INFO     2018-03-28 12:22:58,404 module.py:835] default: "GET /_ah/warmup HTTP/1.1" 200 2
INFO     2018-03-28 12:22:58,480 module.py:835] default: "POST /_tasks/deferred HTTP/1.1" 200 4
INFO     2018-03-28 12:23:00,671 module.py:835] default: "GET / HTTP/1.1" 200 7962
INFO     2018-03-28 12:23:00,732 module.py:835] default: "GET /static/css/ignite.min.css HTTP/1.1" 404 -
INFO     2018-03-28 12:23:00,732 module.py:835] default: "GET /static/js/jquery-3.2.1.min.js HTTP/1.1" 304 -
INFO     2018-03-28 12:23:00,732 module.py:835] default: "GET /static/js/app.js HTTP/1.1" 200 88
INFO     2018-03-28 12:23:00,733 module.py:835] default: "GET /static/css/style.min.css HTTP/1.1" 304 -
^CINFO     2018-03-28 12:23:17,713 shutdown.py:45] Shutting down.
INFO     2018-03-28 12:23:17,714 stub_util.py:360] Applying all pending transactions and saving the datastore
INFO     2018-03-28 12:23:17,719 stub_util.py:363] Saving search indexes
neo@falcon /tmp/v2 $

I tested to call http://localhost:8080 on both installations, and it worked as expected.

We should throw out at least one of these scripts! There needs to be a consistent way to initialize a fresh project.

No, this is exactly the thing we don't want. We provide any way to easily use ViUR, both in the git-based repository setup and in an entirely independent, one-script setup. Surely, with some future developments like ViUR control, the setup.py script might be abandoned, but currently, it is not and shall be kept, especially because the entire documentation depends on it.

So long, Jan

phorward commented 6 years ago

I think this can be closed now :100:

viur-framework / viur-base

mksetup: resulting setup.py not working #4