matthew-sochor-zz / fish.io.ai

Krillin' it since 2017
MIT License
4 stars 3 forks source link

Db backend #38

Closed thenomemac closed 7 years ago

thenomemac commented 7 years ago

database and queue both backed by sqlite3 greatly improved performance as a result. hashing of file names

matthew-sochor-zz commented 7 years ago

Not entirely sure what I'm doing wrong. Although the networking and architecture of this project has jumped way beyond me.

I launched the app with the cli (after installing the env with the prod_install script). App is viewable, however the scoring is on an infinite loop with no progress ever being made.

Now sure what I'm missing.

thenomemac commented 7 years ago

hmm we'll have to work through that infite loop but i can almost gurantee what happend is your scoring process crashed while launching. try running tail -f logs/scoring/fish_pics.log to see the launch log of that process

thenomemac commented 7 years ago

You mean for dev? The prod reqs are a bit more complex.

Ill try to get a script to reproduce the environment but not everything is pip installable. Long run I'd like to use docker. For no I'm focused on getting those two featues added to app around human in the loop labeling. Can we loop at you Mac tomorrow morning to debug issues.

On Apr 10, 2017 10:07 PM, "Matthew A Sochor" notifications@github.com wrote:

can you do a pip freeze to make a requirements?

it'll be easier to make the "required environment"

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/matthew-sochor/fish.io.ai/pull/38#issuecomment-293128980, or mute the thread https://github.com/notifications/unsubscribe-auth/AOeuWt34MirYznamLT3pakPV8eMW1Vdeks5ruuBbgaJpZM4M5gCV .

matthew-sochor-zz commented 7 years ago

I think I figured it out, I'm missing the actual model weights... putting those in now to see if this will run...

thenomemac commented 7 years ago

Fyi the code does work the environment just isn't very reproducible. The this branch was launched via the CLI and is running in prod on the web.

On Apr 10, 2017 10:49 PM, "Josiah Olson" thenomemac@gmail.com wrote:

You mean for dev? The prod reqs are a bit more complex.

Ill try to get a script to reproduce the environment but not everything is pip installable. Long run I'd like to use docker. For no I'm focused on getting those two featues added to app around human in the loop labeling. Can we loop at you Mac tomorrow morning to debug issues.

On Apr 10, 2017 10:07 PM, "Matthew A Sochor" notifications@github.com wrote:

can you do a pip freeze to make a requirements?

it'll be easier to make the "required environment"

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/matthew-sochor/fish.io.ai/pull/38#issuecomment-293128980, or mute the thread https://github.com/notifications/unsubscribe-auth/AOeuWt34MirYznamLT3pakPV8eMW1Vdeks5ruuBbgaJpZM4M5gCV .

thenomemac commented 7 years ago

Try running "source activate fisher" before running the CLI that could be the problem

On Apr 10, 2017 10:49 PM, "Josiah Olson" thenomemac@gmail.com wrote:

You mean for dev? The prod reqs are a bit more complex.

Ill try to get a script to reproduce the environment but not everything is pip installable. Long run I'd like to use docker. For no I'm focused on getting those two featues added to app around human in the loop labeling. Can we loop at you Mac tomorrow morning to debug issues.

On Apr 10, 2017 10:07 PM, "Matthew A Sochor" notifications@github.com wrote:

can you do a pip freeze to make a requirements?

it'll be easier to make the "required environment"

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/matthew-sochor/fish.io.ai/pull/38#issuecomment-293128980, or mute the thread https://github.com/notifications/unsubscribe-auth/AOeuWt34MirYznamLT3pakPV8eMW1Vdeks5ruuBbgaJpZM4M5gCV .

thenomemac commented 7 years ago

edit: source activate fishr

matthew-sochor-zz commented 7 years ago

it was the missing weights!

matthew-sochor-zz commented 7 years ago

Looks good, There is a bug with the logging however. It appears you may need to more specifically cast some things as strings:


Arguments: (['carp', 'walleye', 'white_perch', 'yellow_perch'],)
--- Logging error ---
Traceback (most recent call last):
  File "/Users/matthewsochor/anaconda/lib/python3.5/logging/__init__.py", line 980, in emit
    msg = self.format(record)
  File "/Users/matthewsochor/anaconda/lib/python3.5/logging/__init__.py", line 830, in format
    return fmt.format(record)
  File "/Users/matthewsochor/anaconda/lib/python3.5/logging/__init__.py", line 567, in format
    record.message = record.getMessage()
  File "/Users/matthewsochor/anaconda/lib/python3.5/logging/__init__.py", line 330, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting```
thenomemac commented 7 years ago

Cool. It looks like not much is getting done but it seems to take a very long time to figure out simple things like queues and data schema validation

On Apr 10, 2017 11:00 PM, "Matthew A Sochor" notifications@github.com wrote:

@matthew-sochor approved this pull request.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/matthew-sochor/fish.io.ai/pull/38#pullrequestreview-32004531, or mute the thread https://github.com/notifications/unsubscribe-auth/AOeuWiRuSk6Uwjmvd192pGW1iNMlTtF6ks5ruuzIgaJpZM4M5gCV .