search

Future-House / paper-qa

High accuracy RAG for answering questions from scientific documents with citations
Apache License 2.0
6.35k stars 601 forks source link

local llm example not working, conflicts with litellm #428

Open macromeer opened 1 month ago

macromeer commented 1 month ago

After I had installed paper-qa


mamba create -n paperqa2 python=3.11
mamba activate paperqa2
git clone https://github.com/Future-House/paper-qa.git
cd paper-qa
pip install -e .

...and ran the llamafile from https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#quickstart using

~/home/marco/paper-qa/llava-v1.5-7b-q4.llamafile -cb -np 4 -a my-llm-model --embedding

I ran the code given in https://github.com/Future-House/paper-qa?tab=readme-ov-file#locally-hosted

from paperqa import Settings, ask

local_llm_config = dict(
    model_list=[dict(
        model_name="my_llm_model",
        litellm_params=dict(
            model="my-llm-model",
            api_base="http://localhost:8080/v1",
            api_key="sk-no-key-required",
            temperature=0.1,
            frequency_penalty=1.5,
            max_tokens=512,
        ),
    )]
)

# (model_list is supposed to be a list, so I added [ ])

answer = ask(
    "What manufacturing challenges are unique to bispecific antibodies?",
    settings=Settings(
        llm="my-llm-model",
        llm_config=local_llm_config,
        summary_llm="my-llm-model",
        summary_llm_config=local_llm_config,
    ),
)

This gave me

litellm.exceptions.BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=my-llm-model
 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
dosubot[bot] commented 1 month ago

@macromeer Hello there! I'm a bot designed to help you solve bugs, answer questions, and become a contributor. Feel free to ask me anything while waiting for a human maintainer.

It looks like you're encountering an issue with specifying the LLM provider in your configuration. The error message indicates that the LLM provider is not being correctly specified.

To resolve this, you need to ensure that the litellm_params includes the correct provider information. Here's an updated version of your code with the necessary adjustments:

from paperqa import Settings, ask

local_llm_config = dict(
    model_list=[
        {
            "model_name": "my_llm_model",
            "litellm_params": {
                "model": "my-llm-model",
                "provider": "your-provider-name",  # Add the provider name here
                "api_base": "http://localhost:8080/v1",
                "api_key": "sk-no-key-required",
                "temperature": 0.1,
                "frequency_penalty": 1.5,
                "max_tokens": 512,
            },
        }
    ]
)

answer = ask(
    "What manufacturing challenges are unique to bispecific antibodies?",
    settings=Settings(
        llm="my-llm-model",
        llm_config=local_llm_config,
        summary_llm="my-llm-model",
        summary_llm_config=local_llm_config,
    ),
)

Make sure to replace "your-provider-name" with the actual provider you are using (e.g., "huggingface" if you are using Huggingface models).

This should resolve the BadRequestError by correctly specifying the LLM provider in the litellm_params [1][2].

To continue talking to Dosu, mention @dosu.

mshqn commented 1 month ago

Supporting this. I encounter the same error as above when I run the same code from command line, and I am getting the following error when I run it in the notebook: 

RuntimeError: This event loop is already running
Yi-Wu-3452 commented 1 month ago

Got the same error.

Yi-Wu-3452 commented 1 month ago

Supporting this. I encounter the same error as above when I run the same code from command line, and I am getting the following error when I run it in the notebook:

RuntimeError: This event loop is already running

I did get the same RuntimeError when I use a notebook. You should be good with .py

macromeer commented 1 month ago

I am now trying

vllm serve "meta-llama/Meta-Llama-3-8B-Instruct"

together with

ENDPOINT = "huggingface"
MODEL_NAME = "meta-llama/Meta-Llama-3-8B-Instruct"

my_llm_model = f"vllm/{ENDPOINT}/{MODEL_NAME}"

from paperqa import Settings, ask

local_llm_config = dict(
    model_list=[dict(
        model_name=f"{ENDPOINT}/{MODEL_NAME}",
        litellm_params=dict(
            model=f"{ENDPOINT}/{MODEL_NAME}",
            api_base="http://localhost:8000/v1/chat/completions",
            api_key="sk-no-key-required",
            temperature=0.1,
            #frequency_penalty=1.5,
            max_tokens=512,
        ),
    )]
)

from paperqa.settings import Settings, AgentSettings, AnswerSettings

answer = ask(
    "What is SSIM?",
    settings=Settings(
        llm=f"{ENDPOINT}/{MODEL_NAME}",
        llm_config=local_llm_config,
        summary_llm=f"{ENDPOINT}/{MODEL_NAME}",
        summary_llm_config=local_llm_config,
        paper_directory="/opt/paper-qa/Documents",
        agent=AgentSettings(agent_type="fake"),
        answer=AnswerSettings(evidence_k=3)
    ),
)

print(answer)

on a single paper, but I still do not get a proper answer and some errors:

[15:01:49] Starting paper search for 'Here are three unique keyword searches to help answer the question What is SSIM?:'.                                                                             
           New file to index: s41580-024-00702-6.pdf...                                                                                                                                               

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.
  + Exception Group Traceback (most recent call last):
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
  |     content = await tool._tool_fn(
  |               ^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/tools.py", line 127, in paper_search
  |     index = await get_directory_index(settings=self.settings)
  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/search.py", line 481, in get_directory_index
  |     async with anyio.create_task_group() as tg:
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 685, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
    |     response = await init_response
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 783, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 762, in acompletion
    |     raise HuggingfaceError(
    | litellm.llms.huggingface_restapi.HuggingfaceError: {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/paper-qa/paperqa/agents/search.py", line 377, in process_file
    |     await tmp_docs.aadd(
    |   File "/opt/paper-qa/paperqa/docs.py", line 273, in aadd
    |     result = await llm_model.run_prompt(
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 182, in run_prompt
    |     return await self._run_chat(
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 235, in _run_chat
    |     chunk = await self.achat(messages)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 452, in achat
    |     response = await self.router.acompletion(self.name, messages)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
    |     response = await self.async_function_with_fallbacks(**kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
    |     response = await self.async_function_with_retries(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
    |     response = await original_function(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
    |     response = await _response
    |                ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
    |     result = await original_function(*args, **kwargs)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
    |     raise exception_type(
    |           ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 7465, in exception_type
    |     raise BadRequestError(
    | litellm.exceptions.BadRequestError: litellm.BadRequestError: HuggingfaceException - {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | Received Model Group=huggingface/meta-llama/Meta-Llama-3-8B-Instruct
    | Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
    +------------------------------------
[15:01:50] Starting paper search for 'Image quality assessment, 2010-2024'.                                                                                                                           
           New file to index: s41580-024-00702-6.pdf...                                                                                                                                               

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.
  + Exception Group Traceback (most recent call last):
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
  |     content = await tool._tool_fn(
  |               ^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/tools.py", line 127, in paper_search
  |     index = await get_directory_index(settings=self.settings)
  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/search.py", line 481, in get_directory_index
  |     async with anyio.create_task_group() as tg:
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 685, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
    |     response = await init_response
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 783, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 762, in acompletion
    |     raise HuggingfaceError(
    | litellm.llms.huggingface_restapi.HuggingfaceError: {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/paper-qa/paperqa/agents/search.py", line 377, in process_file
    |     await tmp_docs.aadd(
    |   File "/opt/paper-qa/paperqa/docs.py", line 273, in aadd
    |     result = await llm_model.run_prompt(
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 182, in run_prompt
    |     return await self._run_chat(
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 235, in _run_chat
    |     chunk = await self.achat(messages)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 452, in achat
    |     response = await self.router.acompletion(self.name, messages)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
    |     response = await self.async_function_with_fallbacks(**kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
    |     response = await self.async_function_with_retries(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
    |     response = await original_function(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
    |     response = await _response
    |                ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
    |     result = await original_function(*args, **kwargs)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
    |     raise exception_type(
    |           ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 7465, in exception_type
    |     raise BadRequestError(
    | litellm.exceptions.BadRequestError: litellm.BadRequestError: HuggingfaceException - {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | Received Model Group=huggingface/meta-llama/Meta-Llama-3-8B-Instruct
    | Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
    +------------------------------------
[15:01:51] Starting paper search for 'Structural similarity index, 2000-2024'.                                                                                                                        
           New file to index: s41580-024-00702-6.pdf...                                                                                                                                               

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.
  + Exception Group Traceback (most recent call last):
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
  |     content = await tool._tool_fn(
  |               ^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/tools.py", line 127, in paper_search
  |     index = await get_directory_index(settings=self.settings)
  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/search.py", line 481, in get_directory_index
  |     async with anyio.create_task_group() as tg:
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 685, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
    |     response = await init_response
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 783, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 762, in acompletion
    |     raise HuggingfaceError(
    | litellm.llms.huggingface_restapi.HuggingfaceError: {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/paper-qa/paperqa/agents/search.py", line 377, in process_file
    |     await tmp_docs.aadd(
    |   File "/opt/paper-qa/paperqa/docs.py", line 273, in aadd
    |     result = await llm_model.run_prompt(
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 182, in run_prompt
    |     return await self._run_chat(
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 235, in _run_chat
    |     chunk = await self.achat(messages)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 452, in achat
    |     response = await self.router.acompletion(self.name, messages)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
    |     response = await self.async_function_with_fallbacks(**kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
    |     response = await self.async_function_with_retries(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
    |     response = await original_function(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
    |     response = await _response
    |                ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
    |     result = await original_function(*args, **kwargs)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
    |     raise exception_type(
    |           ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 7465, in exception_type
    |     raise BadRequestError(
    | litellm.exceptions.BadRequestError: litellm.BadRequestError: HuggingfaceException - {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | Received Model Group=huggingface/meta-llama/Meta-Llama-3-8B-Instruct
    | Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
    +------------------------------------
[15:01:52] Starting paper search for 'Peak signal-to-noise ratio, 1990-2024'.                                                                                                                         
           New file to index: s41580-024-00702-6.pdf...                                                                                                                                               

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.
  + Exception Group Traceback (most recent call last):
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
  |     content = await tool._tool_fn(
  |               ^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/tools.py", line 127, in paper_search
  |     index = await get_directory_index(settings=self.settings)
  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/search.py", line 481, in get_directory_index
  |     async with anyio.create_task_group() as tg:
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 685, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
    |     response = await init_response
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 783, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 762, in acompletion
    |     raise HuggingfaceError(
    | litellm.llms.huggingface_restapi.HuggingfaceError: {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/paper-qa/paperqa/agents/search.py", line 377, in process_file
    |     await tmp_docs.aadd(
    |   File "/opt/paper-qa/paperqa/docs.py", line 273, in aadd
    |     result = await llm_model.run_prompt(
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 182, in run_prompt
    |     return await self._run_chat(
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 235, in _run_chat
    |     chunk = await self.achat(messages)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 452, in achat
    |     response = await self.router.acompletion(self.name, messages)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
    |     response = await self.async_function_with_fallbacks(**kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
    |     response = await self.async_function_with_retries(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
    |     response = await original_function(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
    |     response = await _response
    |                ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
    |     result = await original_function(*args, **kwargs)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
    |     raise exception_type(
    |           ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 7465, in exception_type
    |     raise BadRequestError(
    | litellm.exceptions.BadRequestError: litellm.BadRequestError: HuggingfaceException - {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | Received Model Group=huggingface/meta-llama/Meta-Llama-3-8B-Instruct
    | Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
    +------------------------------------
[15:01:53] Starting paper search for 'These searches cover a range of years, from the early 1990s to the present, and use different keywords to capture various aspects of the concept of SSIM. The   
           first search is broad and'.                                                                                                                                                                
           New file to index: s41580-024-00702-6.pdf...                                                                                                                                               

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool paper_search.
  + Exception Group Traceback (most recent call last):
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
  |     content = await tool._tool_fn(
  |               ^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/tools.py", line 127, in paper_search
  |     index = await get_directory_index(settings=self.settings)
  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/paper-qa/paperqa/agents/search.py", line 481, in get_directory_index
  |     async with anyio.create_task_group() as tg:
  |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 685, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
    |     response = await init_response
    |                ^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 783, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 762, in acompletion
    |     raise HuggingfaceError(
    | litellm.llms.huggingface_restapi.HuggingfaceError: {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/paper-qa/paperqa/agents/search.py", line 377, in process_file
    |     await tmp_docs.aadd(
    |   File "/opt/paper-qa/paperqa/docs.py", line 273, in aadd
    |     result = await llm_model.run_prompt(
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 182, in run_prompt
    |     return await self._run_chat(
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 235, in _run_chat
    |     chunk = await self.achat(messages)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/paper-qa/paperqa/llms.py", line 452, in achat
    |     response = await self.router.acompletion(self.name, messages)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
    |     response = await self.async_function_with_fallbacks(**kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
    |     response = await self.async_function_with_retries(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
    |     raise original_exception
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
    |     response = await original_function(*args, **kwargs)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
    |     response = await _response
    |                ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
    |     result = await original_function(*args, **kwargs)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
    |     raise exception_type(
    |           ^^^^^^^^^^^^^^^
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
    |     raise e
    |   File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 7465, in exception_type
    |     raise BadRequestError(
    | litellm.exceptions.BadRequestError: litellm.BadRequestError: HuggingfaceException - {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>user<|end_header_id|>\\n\\nProvide the citation for the following text in MLA Format. Do not write an introductory sentence. If reporting date accessed, the current year is 2024\\n\\nnature reviews molecular cell biology\\nhttps://doi.org/10.1038/s41580-024-00702-6\\nReview article\\n Check for updates\\nLive-cell imaging powered \\nby computation\\nHari Shroff1, Ilaria Testa\\u2009\\n\\u200a\\u20092, Florian Jug\\u2009\\n\\u200a\\u20093 & Suliana Manley\\u2009\\n\\u200a\\u20094\\u2009\\nAbstract\\nSections\\nIntroduction\\nImproving the SNR\\nImproving spatial resolution\\nImproving temporal resolution\\nMulti-colour imaging\\nDiscussion and outlook\\nThe proliferation of microscopy methods for live-cell imaging offers \\nmany new possibilities for users but can also be challenging to \\nnavigate. The prevailing challenge in live-cell fluorescence microscopy \\nis capturing intra-cellular dynamics while preserving cell viability. \\nComputational methods can help to address this challenge and are now \\nshifting the boundaries of what is possible to capture in living systems. \\nIn this Review, we discuss these computational methods focusing on \\nartificial intelligence-based approaches that can be layered on top of \\ncommonly used existing microscopies as well as hybrid methods that \\nintegrate computation and microscope hardware. We specifically \\ndiscuss how computational approaches can improve the signal-to-noise \\nratio, spatial resolution, temporal resolution and multi-colour capacity \\nof live-cell imaging.\\n1Janelia Research Campus, Howard Hughes Medical Institute (HHMI), Ashburn, VA, USA. 2Department of Applied \\nPhysics and Science for Life Laboratory, KTH Royal Institute of Technology, Stockholm, Sweden. 3Fondazione \\nHuman Technopole (HT), Milan, Italy. 4Institute of Physics, School of Basic Sciences, Swiss Federal Institute of \\nTechnology Lausanne (EPFL), Lausanne, Switzerland. \\n\\u2009e-mail: suliana.manley@epfl.ch\\nNature Reviews Molecular Cell Biology | Volume 25 | June 2024 | 443–463\\n443\\nReview article\\nimportant for live fluorescence imaging: SNR, spatial resolution, tem-\\nporal resolution and multi-colour acquisition. We review computa-\\ntional tools that can be easily combined with existing microscopies \\nor used to create hybrid methods that integrate computation and \\nmicroscope hardware (Fig.\\xa01a). We define technical terms and summa-\\nrize the methods reviewed as well as their advantages, requirements, \\npotential artefacts, how they were demonstrated, and links to their \\nsoftware (Table\\xa01). Throughout, we distinguish between tools that \\nare at ready-to-use versus proof-of-concept stage and suggest new \\nopportunities ripe for exploration.\\nIntroduction\\nAn essential feature of living systems is that they are dynamic. Molecules \\nform complexes at specific locations and times, initiating and propagat-\\ning cascades of reactions. Organelles change size, shape, composition \\nand organization within the cell, exchanging signals and molecules as \\nthey contact each other. Cells migrate, divide and differentiate to special-\\nize within a tissue. Thus, sequences of events encode causal relationships \\nand reveal mechanistic details, from the molecular to organismal scale. \\nFluorescence microscopy, which enables dynamic imaging of specific \\nmolec\\n\\nCitation:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
    | Received Model Group=huggingface/meta-llama/Meta-Llama-3-8B-Instruct
    | Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
    +------------------------------------
Failed to execute tool call for tool gather_evidence.
Traceback (most recent call last):
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
    content = await tool._tool_fn(
              ^^^^^^^^^^^^^^^^^^^^
  File "/opt/paper-qa/paperqa/agents/tools.py", line 188, in gather_evidence
    raise EmptyDocsError("Not gathering evidence due to having no papers.")
paperqa.agents.tools.EmptyDocsError: Not gathering evidence due to having no papers.
[15:01:55] Generating answer for 'What is SSIM?'.                                                                                                                                                     

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

Failed to execute tool call for tool gen_answer.
Traceback (most recent call last):
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 427, in acompletion
    response = await init_response
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 783, in acompletion
    raise e
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/llms/huggingface_restapi.py", line 762, in acompletion
    raise HuggingfaceError(
litellm.llms.huggingface_restapi.HuggingfaceError: {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nAnswer in a direct and concise tone. Your audience is an expert, so be highly specific. If there are ambiguous terms or acronyms, first define them.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nAnswer the question below with the context.\\n\\nContext (with relevance scores):\\n\\n\\n\\nValid keys: \\n\\n----\\n\\nQuestion: What is SSIM?\\n\\nWrite an answer based on the context. If the context provides insufficient information reply \"I cannot answer.\"For each part of your answer, indicate which sources most support it via citation keys at the end of sentences, like (Example2012Example pages 3-4). Only cite from the context below and only use the valid keys. Write in the style of a Wikipedia article, with concise sentences and coherent paragraphs. The context comes from a variety of sources and is only a summary, so there may inaccuracies or ambiguities. If quotes are present and relevant, use them in the answer. This answer will go directly onto Wikipedia, so do not add any extraneous information.\\n\\nAnswer (about 200 words, but can be longer):<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nAnswer in a direct and concise tone. Your audience is an expert, so be highly specific. If there are ambiguous terms or acronyms, first define them.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nAnswer the question below with the context.\\n\\nContext (with relevance scores):\\n\\n\\n\\nValid keys: \\n\\n----\\n\\nQuestion: What is SSIM?\\n\\nWrite an answer based on the context. If the context provides insufficient information reply \"I cannot answer.\"For each part of your answer, indicate which sources most support it via citation keys at the end of sentences, like (Example2012Example pages 3-4). Only cite from the context below and only use the valid keys. Write in the style of a Wikipedia article, with concise sentences and coherent paragraphs. The context comes from a variety of sources and is only a summary, so there may inaccuracies or ambiguities. If quotes are present and relevant, use them in the answer. This answer will go directly onto Wikipedia, so do not add any extraneous information.\\n\\nAnswer (about 200 words, but can be longer):<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nAnswer in a direct and concise tone. Your audience is an expert, so be highly specific. If there are ambiguous terms or acronyms, first define them.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nAnswer the question below with the context.\\n\\nContext (with relevance scores):\\n\\n\\n\\nValid keys: \\n\\n----\\n\\nQuestion: What is SSIM?\\n\\nWrite an answer based on the context. If the context provides insufficient information reply \"I cannot answer.\"For each part of your answer, indicate which sources most support it via citation keys at the end of sentences, like (Example2012Example pages 3-4). Only cite from the context below and only use the valid keys. Write in the style of a Wikipedia article, with concise sentences and coherent paragraphs. The context comes from a variety of sources and is only a summary, so there may inaccuracies or ambiguities. If quotes are present and relevant, use them in the answer. This answer will go directly onto Wikipedia, so do not add any extraneous information.\\n\\nAnswer (about 200 words, but can be longer):<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/aviary/env.py", line 197, in _exec_tool_call
    content = await tool._tool_fn(
              ^^^^^^^^^^^^^^^^^^^^
  File "/opt/paper-qa/paperqa/agents/tools.py", line 250, in gen_answer
    state.answer = await state.docs.aquery(
                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/paper-qa/paperqa/docs.py", line 703, in aquery
    answer_result = await llm_model.run_prompt(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/paper-qa/paperqa/llms.py", line 182, in run_prompt
    return await self._run_chat(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/paper-qa/paperqa/llms.py", line 235, in _run_chat
    chunk = await self.achat(messages)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/paper-qa/paperqa/llms.py", line 452, in achat
    response = await self.router.acompletion(self.name, messages)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 735, in acompletion
    raise e
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 723, in acompletion
    response = await self.async_function_with_fallbacks(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3039, in async_function_with_fallbacks
    raise original_exception
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 2893, in async_function_with_fallbacks
    response = await self.async_function_with_retries(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3170, in async_function_with_retries
    raise original_exception
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 3083, in async_function_with_retries
    response = await original_function(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 874, in _acompletion
    raise e
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/router.py", line 846, in _acompletion
    response = await _response
               ^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1585, in wrapper_async
    raise e
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 1405, in wrapper_async
    result = await original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/main.py", line 449, in acompletion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 8187, in exception_type
    raise e
  File "/opt/miniforge3/envs/paperqa2/lib/python3.11/site-packages/litellm/utils.py", line 7465, in exception_type
    raise BadRequestError(
litellm.exceptions.BadRequestError: litellm.BadRequestError: HuggingfaceException - {"object":"error","message":"[{'type': 'missing', 'loc': ('body', 'messages'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nAnswer in a direct and concise tone. Your audience is an expert, so be highly specific. If there are ambiguous terms or acronyms, first define them.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nAnswer the question below with the context.\\n\\nContext (with relevance scores):\\n\\n\\n\\nValid keys: \\n\\n----\\n\\nQuestion: What is SSIM?\\n\\nWrite an answer based on the context. If the context provides insufficient information reply \"I cannot answer.\"For each part of your answer, indicate which sources most support it via citation keys at the end of sentences, like (Example2012Example pages 3-4). Only cite from the context below and only use the valid keys. Write in the style of a Wikipedia article, with concise sentences and coherent paragraphs. The context comes from a variety of sources and is only a summary, so there may inaccuracies or ambiguities. If quotes are present and relevant, use them in the answer. This answer will go directly onto Wikipedia, so do not add any extraneous information.\\n\\nAnswer (about 200 words, but can be longer):<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'missing', 'loc': ('body', 'model'), 'msg': 'Field required', 'input': {'inputs': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nAnswer in a direct and concise tone. Your audience is an expert, so be highly specific. If there are ambiguous terms or acronyms, first define them.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nAnswer the question below with the context.\\n\\nContext (with relevance scores):\\n\\n\\n\\nValid keys: \\n\\n----\\n\\nQuestion: What is SSIM?\\n\\nWrite an answer based on the context. If the context provides insufficient information reply \"I cannot answer.\"For each part of your answer, indicate which sources most support it via citation keys at the end of sentences, like (Example2012Example pages 3-4). Only cite from the context below and only use the valid keys. Write in the style of a Wikipedia article, with concise sentences and coherent paragraphs. The context comes from a variety of sources and is only a summary, so there may inaccuracies or ambiguities. If quotes are present and relevant, use them in the answer. This answer will go directly onto Wikipedia, so do not add any extraneous information.\\n\\nAnswer (about 200 words, but can be longer):<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n', 'parameters': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}, 'stream': False}}, {'type': 'extra_forbidden', 'loc': ('body', 'inputs'), 'msg': 'Extra inputs are not permitted', 'input': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nAnswer in a direct and concise tone. Your audience is an expert, so be highly specific. If there are ambiguous terms or acronyms, first define them.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nAnswer the question below with the context.\\n\\nContext (with relevance scores):\\n\\n\\n\\nValid keys: \\n\\n----\\n\\nQuestion: What is SSIM?\\n\\nWrite an answer based on the context. If the context provides insufficient information reply \"I cannot answer.\"For each part of your answer, indicate which sources most support it via citation keys at the end of sentences, like (Example2012Example pages 3-4). Only cite from the context below and only use the valid keys. Write in the style of a Wikipedia article, with concise sentences and coherent paragraphs. The context comes from a variety of sources and is only a summary, so there may inaccuracies or ambiguities. If quotes are present and relevant, use them in the answer. This answer will go directly onto Wikipedia, so do not add any extraneous information.\\n\\nAnswer (about 200 words, but can be longer):<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n'}, {'type': 'extra_forbidden', 'loc': ('body', 'parameters'), 'msg': 'Extra inputs are not permitted', 'input': {'temperature': 0.1, 'stream': False, 'max_new_tokens': 512, 'details': True, 'return_full_text': False}}]","type":"BadRequestError","param":null,"code":400}
Received Model Group=huggingface/meta-llama/Meta-Llama-3-8B-Instruct
Available Model Group Fallbacks=None LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
[15:01:56] Answer:                                                                                                                                                                                    
answer=Answer(id=UUID('0c9803f3-31ed-44e4-b429-397698fa660e'), question='What is SSIM?', answer='', context='', contexts=[], references='', formatted_answer='', cost=0.0, token_counts={}, config_md5='53d48adb6aaf94ae410e6c7570060390', used_contexts=set()) bibtex=None status=<AgentStatus.SUCCESS: 'success'> timing_info=None duration=0.0 stats=None