Open kiraseed opened 5 years ago
pkgs_to_install
?@rkooo567
def __get_data_lev(dataframe, id_level): dg = dataframe.groupby(id_level) dataframe = dg.first() return dataframe if i call the function , the result show error that 'Failed to retrieve a prediction response within the specified latency SLO'. if i don't call it ,the result is correct. how to call this kind of function ?
Hi, @kiraseed . I think that your closure function might be wrong. Would you please share your all codes with us?
The returned type need to be a list of strings. Not dataframe type.
On Wed, Jul 17, 2019 at 11:23 PM Sungjun.Kim notifications@github.com wrote:
Hi, @kiraseed https://github.com/kiraseed . I think that your closure function might be wrong. Would you please share your all codes with us?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ucbrise/clipper/issues/744?email_source=notifications&email_token=AFBD7AYBBNFK2JIF2SH3ZVTQAAD63A5CNFSM4IENHTZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HOOFY#issuecomment-512681751, or mute the thread https://github.com/notifications/unsubscribe-auth/AFBD7AZ3UFANBTCH6NR2NNLQAAD63ANCNFSM4IENHTZA .
@withsmilo @rkooo567 thank you for your suggestions. now the code can run, but requests.post must run tow times. the first one show error : 'default_explanation': 'Failed to retrieve a prediction response within the specified latency SLO', the second one can output data, why is it?
Clipper batch queries and adaptively adjust the max batch size to maximize throughput within the specified SLO requirement. If it cannot return the query by given SLO (which you provide in your input), it returns that default query result (the one you are looking at). So, it is most likely that your SLO specification is too low compared to your environment / complexity of code. (So, I guess in the first query, you couldn't return the result within the specified SLO, but second query could.)
Can you try to increase slo_micros
argument's value and try again?
@rkooo567 i increase slo_micros's value , but can't solve the problem.
Hmm okay. Can you do docker ps
, do docker logs
on the model containers/query frontend, and show us logs for both of them? (Let me know if you are using KubernetesContainerManager
)
i use DockerContainerManager; docker ps: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1bbbd89a0872 default-cluster-sum2:1 "/container/containe…" About a minute ago Up About a minute (healthy) sum2_1-69145 d0e24ed41026 prom/prometheus:v2.9.2 "/bin/prometheus --c…" 2 minutes ago Up 2 minutes 0.0.0.0:9090->9090/tcp metric_frontend-71308 0e1cd6b2a80f clipper/frontend-exporter:0.4.1 "python /usr/src/app…" 2 minutes ago Up 2 minutes query_frontend_exporter-26407 66369f2502bf clipper/query_frontend:0.4.1 "/clipper/query_fron…" 2 minutes ago Up 2 minutes 0.0.0.0:1337->1337/tcp, 0.0.0.0:7000->7000/tcp query_frontend-26407 653c6d5d3673 clipper/management_frontend:0.4.1 "/clipper/mgmt_front…" 2 minutes ago Up 2 minutes 0.0.0.0:1338->1338/tcp mgmt_frontend-97195 511529faa8bd redis:alpine "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:6379->6379/tcp redis-4224
clipper_conn.get_all_apps(verbose=True)
docker logs: "docker logs" requires exactly 1 argument. See 'docker logs --help'.
Usage: docker logs [OPTIONS] CONTAINER
Fetch the logs of a container
Hi @kiraseed! First, please run docker ps -a
. This will give you something like the following:
Look for the container where the image
field matches the name of the model you deployed. For example, if you gave the name mymodel
and version 1
to the deploy function, the image name should be mymodel:1
.
Copy the container ID, then run docker logs <Container ID>
where <Container ID>
is the container id you copied earlier. If you could post the output of that command here, that would help me debug the issue!
Thanks @RehanSD the output is: /bin/sh: 1: Syntax error: end of file unexpected;
Hmm. That's odd. Are you using a custom dockerfile? Could you also post a screens hot of the output of docker ps -a
and specify which image you used as input for docker logs
?
i have not used custom dockerfile. docker ps -a all output is:
CONTAINER ID | IMAGE | COMMAND | CREATED | STATUS | PORTS | NAMES |
---|---|---|---|---|---|---|
474034ecf562 | default-cluster-sum2:1 | /container/containe… | About a minute ago | Up About a minute(healthy) | sum2_1-29302 | |
6d681794ee19 | prom/prometheus:v2.9.2 | /bin/prometheus --c… | About a minute ago | Up About a minute | 0.0.0.0:9090->9090/tcp | metric_frontend-82350 |
198a52a937b7 | clipper/frontend-exporter:0.4.1 | python /usr/src/app… | About a minute ago | Up About a minute | query_frontend_exporter-81119 | |
a5d791f607ed | clipper/query_frontend:0.4.1 | /clipper/query_fron… | About a minute ago | Up About a minute | 0.0.0.0:1337->1337/tcp,0.0.0.0:7000->7000/tcp | query_frontend-81119 |
bcd2638cc55c | clipper/management_frontend:0.4.1 | /clipper/mgmt_front… | About a minute ago | Up About a minute | 0.0.0.0:1338->1338/tcp | mgmt_frontend-46278 |
49cdc0de48a9 | redis:alpine | docker-entrypoint.s… | About a minute ago | Up About a minute | 0.0.0.0:6379->6379/tcp | redis-50798 |
docker logs of them are the same problem that /bin/sh: 1: Syntax error: end of file unexpected.
This error is really odd. Is it the logs of docker logs 474034ecf562
? It is also weird that the container is not broken although there is an error. Also, Would you mind sharing your prediction function? with clipper code? I will try to reproduce the problem on my local machine.
when i deploy python function that contain pandas' package, the clipper output error, does the cliper not support by padans (eg: grouby, merge)? or how to solve it?