Build error when using ASM on local server : recipe for target 'schema' failed

[julian-poidevin]

Hello,

I updated my ASM fork with the latest changes and I have an error during build at the schema.py step, here's the error log :

root@vps759328:~/repo/asm3# make all
[compile javascript] =================
cd scripts/jslint && ./run.py
accounts.js
accounts_trx.js
additional.js
animal.js
animal_bulk.js
animal_costs.js
animal_diet.js
animal_find.js
animal_find_results.js
animal_new.js
animal_view_adoptable.js
batch.js
calendarview.js
change_password.js
change_user_settings.js
citations.js
clinic_appointment.js
clinic_calendar.js
clinic_invoice.js
common.js
common_animalchooser.js
common_animalchoosermulti.js
common_map.js
common_personchooser.js
common_tableform.js
common_widgets.js
(node:17809) [DEP0027] DeprecationWarning: util.puts is deprecated. Use console.log instead.
176:25:WARNING:'rv' was used before it was defined.
184:33:WARNING:'rv' was used before it was defined.
185:34:WARNING:'rv' was used before it was defined.

csvexport.js
csvimport.js
csvimport_paypal.js
diary.js
diarytask.js
diarytasks.js
document_edit.js
document_repository.js
document_templates.js
donation_receive.js
donations.js
giftaid_hmrc_spreadsheet.js
header.js
header_additional.js
header_edit_header.js
htmltemplates.js
incident.js
incident_find.js
incident_find_results.js
incident_map.js
incident_new.js
latency.js
licence.js
licence_renewal.js
litters.js
log.js
log_new.js
login.js
lookups.js
lostfound.js
lostfound_find.js
lostfound_find_results.js
lostfound_new.js
mailmerge.js
mailmerge_criteria.js
main.js
media.js
medical.js
medicalprofile.js
mobile.js
mobile_sign.js
move_adopt.js
move_deceased.js
move_foster.js
move_gendoc.js
move_reclaim.js
move_reserve.js
move_retailer.js
move_transfer.js
movements.js
onlineform.js
onlineform_extra.js
onlineform_incoming.js
onlineforms.js
options.js
person.js
person_find.js
person_find_results.js
person_investigation.js
person_links.js
person_new.js
publish.js
publish_logs.js
publish_options.js
report.js
report_criteria.js
report_export.js
report_images.js
reports.js
roles.js
rota.js
search.js
shelterview.js
sql.js
staff_rota.js
stocklevel.js
task.js
test.js
timeline.js
transport.js
traploan.js
users.js
vaccination.js
vouchers.js
waitinglist.js
waitinglist_new.js
waitinglist_results.js
[compile python] =====================
flake8 --config=scripts/flake8 src/*.py src/asm3/*.py src/asm3/dbms/*.py src/asm3/publishers/*.py
[compile jsmin] ======================
gcc -o scripts/jsmin/jsmin scripts/jsmin/jsmin.c
[clean] ============================
rm -f src/static/js/min/*.min.js
rm -f cscope*
rm -f tags
rm -f src/*.pyc
rm -rf src/__pycache__
rm -f src/asm3/*.pyc
rm -rf src/asm3/__pycache__
rm -f src/asm3/dbms/*.pyc
rm -rf src/asm3/dbms/__pycache__
rm -f src/asm3/locales/*.pyc
rm -rf src/asm3/locales/__pycache__
rm -f src/asm3/pbkdf2/*.pyc
rm -rf src/asm3/pbkdf2/__pycache__
rm -f src/asm3/publishers/*.pyc
rm -rf src/asm3/publishers/__pycache__
[tags] ============================
rm -f tag
ctags -f tags src/*.py src/asm3/*.py src/asm3/publishers/*.py src/asm3/dbms/*.py src/static/js/*.js
ctags: Warning: ignoring null tag in src/static/js/common.js
# Generate minified versions of all javascript in min folder
[minify] =============================
mkdir -p src/static/js/min
for i in src/static/js/*.js; do echo $i; cat $i | scripts/jsmin/jsmin > src/static/js/min/`basename $i .js`.min.js; done
src/static/js/accounts.js
src/static/js/accounts_trx.js
src/static/js/additional.js
src/static/js/animal.js
src/static/js/animal_bulk.js
src/static/js/animal_costs.js
src/static/js/animal_diet.js
src/static/js/animal_find.js
src/static/js/animal_find_results.js
src/static/js/animal_new.js
src/static/js/animal_view_adoptable.js
src/static/js/batch.js
src/static/js/calendarview.js
src/static/js/change_password.js
src/static/js/change_user_settings.js
src/static/js/citations.js
src/static/js/clinic_appointment.js
src/static/js/clinic_calendar.js
src/static/js/clinic_invoice.js
src/static/js/common.js
src/static/js/common_animalchooser.js
src/static/js/common_animalchoosermulti.js
src/static/js/common_map.js
src/static/js/common_personchooser.js
src/static/js/common_tableform.js
src/static/js/common_widgets.js
src/static/js/csvexport.js
src/static/js/csvimport.js
src/static/js/csvimport_paypal.js
src/static/js/diary.js
src/static/js/diarytask.js
src/static/js/diarytasks.js
src/static/js/document_edit.js
src/static/js/document_repository.js
src/static/js/document_templates.js
src/static/js/donation_receive.js
src/static/js/donations.js
src/static/js/giftaid_hmrc_spreadsheet.js
src/static/js/header.js
src/static/js/header_additional.js
src/static/js/header_edit_header.js
src/static/js/htmltemplates.js
src/static/js/incident.js
src/static/js/incident_find.js
src/static/js/incident_find_results.js
src/static/js/incident_map.js
src/static/js/incident_new.js
src/static/js/latency.js
src/static/js/licence.js
src/static/js/licence_renewal.js
src/static/js/litters.js
src/static/js/log.js
src/static/js/log_new.js
src/static/js/login.js
src/static/js/lookups.js
src/static/js/lostfound.js
src/static/js/lostfound_find.js
src/static/js/lostfound_find_results.js
src/static/js/lostfound_new.js
src/static/js/mailmerge.js
src/static/js/mailmerge_criteria.js
src/static/js/main.js
src/static/js/media.js
src/static/js/medical.js
src/static/js/medicalprofile.js
src/static/js/mobile.js
src/static/js/mobile_sign.js
src/static/js/move_adopt.js
src/static/js/move_deceased.js
src/static/js/move_foster.js
src/static/js/move_gendoc.js
src/static/js/move_reclaim.js
src/static/js/move_reserve.js
src/static/js/move_retailer.js
src/static/js/move_transfer.js
src/static/js/movements.js
src/static/js/onlineform.js
src/static/js/onlineform_extra.js
src/static/js/onlineform_incoming.js
src/static/js/onlineforms.js
src/static/js/options.js
src/static/js/person.js
src/static/js/person_find.js
src/static/js/person_find_results.js
src/static/js/person_investigation.js
src/static/js/person_links.js
src/static/js/person_new.js
src/static/js/publish.js
src/static/js/publish_logs.js
src/static/js/publish_options.js
src/static/js/report.js
src/static/js/report_criteria.js
src/static/js/report_export.js
src/static/js/report_images.js
src/static/js/reports.js
src/static/js/roles.js
src/static/js/rota.js
src/static/js/search.js
src/static/js/shelterview.js
src/static/js/sql.js
src/static/js/staff_rota.js
src/static/js/stocklevel.js
src/static/js/task.js
src/static/js/test.js
src/static/js/timeline.js
src/static/js/transport.js
src/static/js/traploan.js
src/static/js/users.js
src/static/js/vaccination.js
src/static/js/vouchers.js
src/static/js/waitinglist.js
src/static/js/waitinglist_new.js
src/static/js/waitinglist_results.js
# Generate a rollup file of all javascript files
[rollup] =============================
scripts/rollup/rollup.py > src/static/js/min/rollup.min.js
# Generate a JSON schema of the database for use when editing
# SQL within the program
[schema] =============================
scripts/schema/schema.py > src/static/js/min/schema.min.js
Traceback (most recent call last):
  File "scripts/schema/schema.py", line 15, in <module>
    for table in db.query("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name"):
  File "/usr/lib/python3/dist-packages/web/db.py", line 1069, in query
    out = DB.query(self, *a, **kw)
  File "/usr/lib/python3/dist-packages/web/db.py", line 664, in query
    db_cursor = self._db_cursor()
  File "/usr/lib/python3/dist-packages/web/db.py", line 575, in _db_cursor
    return self.ctx.db.cursor()
  File "/usr/lib/python3/dist-packages/web/db.py", line 516, in _getctx
    self._load_context(self._ctx)
  File "/usr/lib/python3/dist-packages/web/db.py", line 527, in _load_context
    ctx.db = self._connect(self.keywords)
  File "/usr/lib/python3/dist-packages/web/db.py", line 555, in _connect
    return self.db_module.connect(**keywords)
sqlite3.OperationalError: unable to open database file
Makefile:66: recipe for target 'schema' failed
make: *** [schema] Error 1

[bobintetley]

thanks, rv is an accidental global and should not have been there.

[julian-poidevin]

@bobintetley This did not fix the issue unfortunately, I keep having the same build problem. The problem is related to schema.py, I don't get how a change to common_widgets.js fixed the issue 🤔 .

[julian-poidevin]

From what I can tell, for me the build stopped working starting c18ca71ed0449a16e5d098cb52d84bb1135dc123.

Isn't it related to the use of the constant path /home/robin/tmp/asm3.db ? This path does not exists for users using ASM on their own server like me 😃 .

https://github.com/bobintetley/asm3/blob/864786a3d0bc986cd1f6734108c3532dab7758f9/scripts/schema/schema.py#L8

[bobintetley]

You're right, the error I fixed was further up in your output!

Yes, the fault is with schema. I updated things to generate the schema.js file (used for code complete in the SQL editor) statically for performance reasons. I meant to have the build generate the schema database that it uses - but forgot to do this and left it accessing a hard coded file on my machine.

The commit I just added fixes this. Thanks for your vigilance!

[julian-poidevin]

Hello @bobintetley, I just tried to rebuild ASM this morning on my server and the same error is occurring :

# Generate a JSON schema of the database for use when editing
# SQL within the program
[schema] =============================
scripts/schema/schema.py > src/static/js/min/schema.min.js
Traceback (most recent call last):
  File "scripts/schema/schema.py", line 15, in <module>
    for table in db.query("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name"):
  File "/usr/lib/python3/dist-packages/web/db.py", line 1069, in query
    out = DB.query(self, *a, **kw)
  File "/usr/lib/python3/dist-packages/web/db.py", line 664, in query
    db_cursor = self._db_cursor()
  File "/usr/lib/python3/dist-packages/web/db.py", line 575, in _db_cursor
    return self.ctx.db.cursor()
  File "/usr/lib/python3/dist-packages/web/db.py", line 516, in _getctx
    self._load_context(self._ctx)
  File "/usr/lib/python3/dist-packages/web/db.py", line 527, in _load_context
    ctx.db = self._connect(self.keywords)
  File "/usr/lib/python3/dist-packages/web/db.py", line 555, in _connect
    return self.db_module.connect(**keywords)
sqlite3.OperationalError: unable to open database file
Makefile:67: recipe for target 'schema' failed
make: *** [schema] Error 1

Let me know if I can provide any additional details that might help you 😉

[bobintetley]

That's odd, schema.db must be failing, but it doesn't look like the schema target is trying to run it.

What happens when you run "make scripts/schema/schema.db"  ? When I run "make schema" it builds schema.db if it is not present.

[julian-poidevin]

As far as I can see, the scripts is running, the schema.db file is generated :

Here's the log output of make scripts/schema/schema.db command : makeSchemaLog.txt

[bobintetley]

The error is that the schema.py file is failing to open schema.db - probably because I didn't update it and it's still looking for the file on my hard disk! What an idiot. Sorry about that...

[julian-poidevin]

I was wondering why there was no changes to shema.py file but I'm not that familiar with ASM3 architecture so I thought that was normal 😄. No problem, will try this right away !