search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
30.78k stars 4.67k forks source link

[Bug]: Regression ~~for AWQ marlin kernels~~ from v0.6.2 to v0.6.3 when using CUDA Graphs #9417

Closed joennlae closed 2 weeks ago

joennlae commented 1 month ago

Your current environment

First of all: fantastic project :-) Thank you for everything.

I would like to fix this bug. But I just do not have the capacity now. So I just thought I would try to make a good bug report.

Model Input Dumps

No response

🐛 Describe the bug

If I run this model in v0.6.2:

vllm serve hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4 -tp 4 --gpu-memory-utilization 0.90 --max-model-len 32768

All works well and good :-)

If I run it in v0.6.3

vllm serve hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4 -tp 4 --gpu-memory-utilization 0.90 --max-model-len 32768 --enforce-eager

All works well and good with enforce eager :-)

If I drop the enforce-eager

vllm serve hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4 -tp 4 --gpu-memory-utilization 0.90 --max-model-len 32768

I get random repetition on large prompts 6000+ token. Or if I do multiple request in parallel I get CUDA: illegal memory access

My guess is that there is something dynamic in the updated awq_marlin kernels.

My hunch (this is untested): #8973 but I am not fully understanding how my non MoE should be affected by this.

Before submitting a new issue...

leangermany commented 1 month ago

We got the same issue, but it works with enforce-eager too.

simon-mo commented 1 month ago

cc @mgoin to route

mgoin commented 1 month ago

Thanks for reporting, @ElizaWszola do you have any ideas? We added awq_marlin for dense models quite a while ago (https://github.com/vllm-project/vllm/pull/6612), so I wouldn't expect changes between 0.6.2 and 0.6.3

It is possible that some of the binary size reduction work introduced a dynamic case, but I don't remember this applying to non-MoE.

@joennlae @leangermany any minimal reproducible example would be appreciated! I tried the following with and without enforce_eager using Llama 3.1 8B AWQ, and found that the output was completely unaffected.

Script:

from vllm import LLM, SamplingParams

llm = LLM(model="hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4", enforce_eager=True)
messages = [{"role": "user", "content": "Prove that the difference between two consecutive cubes cannot be divisible by 5, using the fact that the only possible remainders when a cube is divided by 5 are 0, 1, and -1."}]
outputs = llm.chat(messages, sampling_params=SamplingParams(temperature=0.0, max_tokens=1000))
print(outputs[0].outputs[0].text)

Output:

To prove that the difference between two consecutive cubes cannot be divisible by 5, we will use the fact that the only possible remainders when a cube is divided by 5 are 0, 1, and -1.

Let's consider two consecutive cubes, n^3 and (n+1)^3. 

We can express n^3 and (n+1)^3 as follows:

n^3 = n(n-1)(n+1) + 3n^2 + n
(n+1)^3 = n^3 + 3n^2 + 3n + 1

Now, let's find the difference between (n+1)^3 and n^3:

(n+1)^3 - n^3 = (n^3 + 3n^2 + 3n + 1) - (n^3 + 3n^2 + n)
= 3n + 1

Now, let's consider the possible remainders when 3n + 1 is divided by 5. 

Since the only possible remainders when a cube is divided by 5 are 0, 1, and -1, we can consider the possible remainders when n is divided by 5:

- If n is divisible by 5, then 3n is divisible by 5 and 3n + 1 is not divisible by 5.
- If n leaves a remainder of 1 when divided by 5, then 3n leaves a remainder of 2 when divided by 5 and 3n + 1 leaves a remainder of 0 when divided by 5.
- If n leaves a remainder of 2 when divided by 5, then 3n leaves a remainder of 1 when divided by 5 and 3n + 1 leaves a remainder of 2 when divided by 5.
- If n leaves a remainder of 3 when divided by 5, then 3n leaves a remainder of 4 when divided by 5 and 3n + 1 leaves a remainder of 0 when divided by 5.
- If n leaves a remainder of 4 when divided by 5, then 3n leaves a remainder of 2 when divided by 5 and 3n + 1 leaves a remainder of 0 when divided by 5.

In all cases, the remainder when 3n + 1 is divided by 5 is not 0, 1, or -1. Therefore, the difference between two consecutive cubes cannot be divisible by 5.
joennlae commented 1 month ago

So, unfortunately, I cannot share the exact prompt as we use sensitive user data.

joennlae commented 1 month ago

Maybe it is not a marlin issue (sorry for the call out): #9448 They report it with a non quantised model.

leangermany commented 1 month ago

Quantization config:

{
  "_name_or_path": "/data/chatbot/vllm/Llama-3.1-SuperNova-Lite,
  "architectures": [
    "LlamaForCausalLM"
  ],
  "attention_bias": false,
  "attention_dropout": 0.0,
  "bos_token_id": 128000,
  "eos_token_id": [
    128001,
    128008,
    128009
  ],
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 14336,
  "max_position_embeddings": 131072,
  "mlp_bias": false,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 8,
  "pretraining_tp": 1,
  "quantization_config": {
    "bits": 4,
    "group_size": 32,
    "modules_to_not_convert": null,
    "quant_method": "awq",
    "version": "gemm",
    "zero_point": true
  },
  "rms_norm_eps": 1e-05,
  "rope_scaling": {
    "factor": 8.0,
    "high_freq_factor": 4.0,
    "low_freq_factor": 1.0,
    "original_max_position_embeddings": 8192,
    "rope_type": "llama3"
  },
  "rope_theta": 500000.0,
  "tie_word_embeddings": false,
  "torch_dtype": "float16",
  "transformers_version": "4.44.0",
  "use_cache": false,
  "vocab_size": 128256
}

Not working example, that answers not correct:

[{
        "role": "system",
        "content": "**Context:**  \nYou are an AI assistent for kohell named kohell assistant. The user is requesting content of a file.\nThe file content matching the user input is wrapped by [DOCUMENTATION_START] and [DOCUMENTATION_END].\n**Your Task:**\nAnalyze the provided content in this context thoroughly.  Answer all questions truthfully based solely on the information within this content. If you cannot find an answer within the content, state that the information is not available. Do not speculate or provide information from external sources.  \n**Formatting:**\nWhen encountering enumerations like 1, 2, 3 or tables in markdown format within the documentation, convert them into HTML for better presentation.  Use ordered lists (<ol>) for numbered enumerations and unordered lists (<ul>) for bullet points that are represented in markdown with dashes followed by a space (- ) or large comma (,) seperated lists. Use HTML tables (<table>) for presenting data in tabular format. Convert markdown syntax to html tags as needed.   \n**Please remember:**\nMaintain a neutral and objective tone throughout the conversation.  Focus on providing clear, concise answers.  You love to help your colleagues out."
    }, {
        "role": "user",
        "content": "[DOCUMENTATION_START]   \n===\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:26}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n\n\n| Cancer Type (Baseline) | n | Percent | Cancer Type (Follow-up) | n | Percent |\n| --- | --- | --- | --- | --- | --- |\n| Breast | 314 | 31.9% | Breast | 221 | 37.4% |\n| Prostate | 94 | 9.8% | Prostate | 48 | 8.1% |\n| Lymphoma | 73 | 7.6% | Lymphoma | 42 | 7.1% |\n| Colon or rectum | 69 | 7.2% | Colon or rectum | 42 | 7.1% |\n| Skin-Melanoma | 56 | 5.9% | Skin-Melanoma | 27 | 4.6% |\n| Skin-Not Melanoma | 48 | 5.0% | Skin-Not Melanoma | 30 | 5.1% |\n| Thyroid | 46 | 4.8% | Thyroid | 26 | 4.4% |\n| Uterine | 35 | 3.7% | Uterine | 15 | 2.5% |\n| Kidney | 26 | 2.7% | Kidney | 16 | 2.7% |\n| Testis | 25 | 2.6% | Testis | 17 | 2.9% |\n| Bladder | 19 | 2.0% | Bladder | 14 | 2.4% |\n| Lung | 13 | 1.4% | Lung | 6 | 1.0% |\n| Other (please specify) | 155 | 15.7% | Other (please specify) | 87 | 14.7% |\n\n\n“Other” Types of Cancer:\n\n\nHead and Neck Cancer: (n = 24)\n\n\nLeukemia: (n = 21)\n\n\nOvarian Cancer: (n = 20) Cervical cancer (n = 13) Sarcoma: (n = 12)\n\n\nTwo or more Types: (n = 9) Pancreatic Cancer: (n = 5) Brain Tumor: (n = 4) Multiple Myeloma: (n = 4) Esophageal cancer: (n = 3) Uterine Cancer: (n = 3) Lung Cancer: (n = 3) Carcinoid (n = 2)\n\n\nLiver Cancer: (n = 2) Ocular Melanoma: (n = 2) Salivary Gland Tumor: (n = 2)\n\n\nAmyloidosis: (n = 1)\n\n\nBlood Cancer: (n = 1)\n\n\nCarcinoma of unknown origin: (n = 1)\n\n\nColon Cancer: (n = 1)\n\n\nGastrointestinal stromal tumor: (n = 1)\n\n\nGestational trophoblastic disease: (n = 1)\n\n\nNasal/Brain Tumor: (n = 1)\n\n\nNeuroblastoma: (n = 1)\n\n\nParaganglioma: (n = 1)\n\n\nPrimary Peritoneal Cancer: (n = 1)\n\n\nSpinal Ependymoma: (n = 1)\n\n\nThymus Cancer: (n = 1)\n\n\nThyroid Cancer: (n = 1)\n\n\nVaginal Cancer: (n = 1)\n\n\nVulvar Cancer: (n = 1)\n\n\n26\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:26}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:30}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nEmployee Survey - Follow-up Data\n\n\nSince July 2014, have you been diagnosed with a kind of cancer that is different from the diagnosis you had then?\n\n\n\n\n| Response - Yes | n | Percent |\n| --- | --- | --- |\n| Skin (not melanoma) | 3 | 0.5% |\n| Lymphoma | 1 | 0.2% |\n| Prostate | 1 | 0.2% |\n| Skin (melanoma) | 1 | 0.2% |\n| Kidney | 1 | 0.2% |\n| Other (please specify) | 6 | 1.0% |\n| Response - No | 579 | 97.8% |\n\n\nOther Reponses:\n\n\nCervical (n = 2)\n\n\nPyriform Sinus/Laryngeal Cancer (n = 1)\n\n\nNeuroendocrine cancer (pancreas liver cancer) (n = 1)\n\n\nStomach (n = 1)\n\n\nOvarian (n = 1)\n\n\n30\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:30}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:89}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 24)    Which kind of cancer diagnosis have you received since July 2014 in addition to the one you had then?\n\n\nO Bladder\n\n\nO Breast\n\n\nO Colon or rectum\n\n\nO Kidney\n\n\nO Lung\n\n\nO Lymphoma\n\n\nO Prostate\n\n\nO Skin (melanoma)\n\n\nO Skin (not melanoma)\n\n\nO Testis (testicular cancer)\n\n\nO Thyroid\n\n\nO Uterine (also known as endometrial cancer)\n\n\nO Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 25)    Since July 2014, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n● □    I have not had any cancer treatment in the last 6 months.\n● □    Oral medication (EXCEPT oral chemotherapy)\n● □    Radiation\n● □    Surgery\n● □    Oral chemotherapy\n● □    Intravenous chemotherapy (may be given through your vein or a through a catheter or mediport)\n● □    Other chemotherapy (This may include chemotherapy given directly into the spinal fluid or into the peritoneal cavity. Note that these kinds of chemotherapy are very rare.)\n● □    Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 26)    Have you disclosed your cancer (or history of cancer) to any supervisor or human resource staff member at the company that sent you the invitation to the first survey?\n\n\nO Yes\n\n\nO No [Skips to question 28]\n\n\n89\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:89}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:28}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nDuring the last 6 months, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n\n\n| Response (Baseline) | n | Percent |\n| --- | --- | --- |\n| I have not had any cancer treatment in the last 6 months. | 566 | 57.5% |\n| Surgery | 168 | 17.1% |\n| Oral medication (EXCEPT oral chemotherapy) | 147 | 14.9% |\n| Radiation | 83 | 8.4% |\n| Intravenous chemotherapy (may be given through your vein or a through a catheter or mediport) | 75 | 7.6% |\n| Oral chemotherapy | 43 | 4.4% |\n| Other chemotherapy (This may include chemotherapy given directly into the spinal fluid or into the peritoneal cavity. Note that these kinds of chemotherapy are very rare.) | 3 | 0.3% |\n| Other (please specify) | 91 | 9.2% |\n\n\nExamples of “Other” kinds of treatment:\n\n\nInjected Medicines (not chemotherapy): (n = 22)\n\n\nScans and/or Imaging Services (full body scan, mammogram, ultrasound, cystoscopy, MRI,\n\n\nCAT Scan, sonogram): (n = 16)\n\n\nFollow-up Visits: (n = 12)\n\n\nSkin Treatments (freezing/burning/biopsies/grafts): (n = 10)\n\n\nBlood Tests: (n = 5)\n\n\nHormone therapy: (n = 4)\n\n\nPhysical Therapy: (n = 4)\n\n\nStem Cell Transplant: (n = 2)\n\n\nDiagnostic Testing: (n = 2)\n\n\nTopical Chemotherapy: (n = 2)\n\n\nInhaler: (n = 1)\n\n\nRadioactive Iodine: (n = 1)\n\n\nParticipant in a Blind Study: (n = 1)\n\n\n28\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:28}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:75}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 23)    How long ago did you receive this cancer diagnosis?\n\n\nO 6 months ago or less\n\n\nO 7 to 12 months ago\n\n\nO 13 to 24 months ago\n\n\nO 2 to 5 years ago\n\n\nO More than 5 years ago\n\n\n● 24)    During the last 6 months, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n● □    I have not had any cancer treatment in the last 6 months.\n● □    Oral medication (EXCEPT oral chemotherapy)\n\n\n□ Radiation\n\n\n● □    Surgery\n● □    Oral chemotherapy\n● □    Intravenous chemotherapy (may be given through your vein or a through a catheter or mediport)\n● □    Other chemotherapy (This may include chemotherapy given directly into the spinal fluid or into the peritoneal cavity. Note that these kinds of chemotherapy are very rare.)\n● □    Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 25)    Have you disclosed your cancer (or history of cancer) to any supervisor or human resource staff member at the company that sent you the survey invitation?\n\n\nO Yes\n\n\nO No [Skip to question 27]\n\n\n75\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:75}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:31}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nSince July 2014, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n\n\n| Response (Follow-up) | n | Percent |\n| --- | --- | --- |\n| None | 374 | 54.8% |\n| Oral (not oral chemo) | 110 | 16.1% |\n| Radiation | 33 | 4.8% |\n| Surgery | 48 | 7.0% |\n| Oral chemotherapy | 22 | 3.2% |\n| IV chemo | 41 | 6.0% |\n| Other chemo | 1 | 0.1% |\n| Other (please specify) | 53 | 7.8% |\n\n\nOther Responses:\n\n\nInjected Medicines (not chemotherapy): (n = 12)\n\n\nFollow-up Visits: (n = 12)\n\n\nScans and/or Imaging Services (full body scan, mammogram, ultrasound, cystoscopy, MRI,\n\n\nCAT Scan, sonogram): (n = 8)\n\n\nSkin Treatments (freezing/burning/biopsies/grafts): (n = 8)\n\n\nPhysical Therapy: (n = 3)\n\n\nBlood Tests: (n = 2)\n\n\nHormone therapy: (n = 2)\n\n\nDiagnostic Testing: (n = 2)\n\n\nTopical Chemotherapy: (n = 2)\n\n\nIncontinence devices: (n = 1)\n\n\nFluoride Treatment: (n = 1)\n\n\n31\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:31}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:27}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nEmployee Survey Report - Baseline Data\n\n\nHow long ago did you receive this cancer diagnosis?\n\n\n\n\n| Response (Baseline) | n | Percent |\n| --- | --- | --- |\n| < 6 mo. | 102 | 10.4% |\n| 7-12 mo. | 107 | 11.0% |\n| 13-24 mo. | 143 | 14.6% |\n| 2-5 yrs. | 291 | 29.8% |\n| > 5 yrs. | 334 | 34.2% |\n\n\n27\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:27}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:48}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nThe eToolkit often suggested other sources of information about cancer and workplace issues. Since receiving access to the eToolkit, have you looked up any of those resources?\n\n\nWhat were some of those resources?\n\n\nAll of them.\n\n\nMy brother has Leukemia and my father was diagnosed yesterday as terminal with another type of blood cancer. I was just reviewing the site and pleased to see the information that exists for those impacted by these horrible diseases.\n\n\nCoping with the effects of Chemo and Radiation Chemo-Brain\n\n\nCaregiver, American Cancer Society\n\n\nI have lost 3 family members in the last 18 months\n\n\nThe links to cancer information regarding side effects of treatment, etc. Can't remember the link.\n\n\nProton Radiation program at Loma Linda Medical Center\n\n\nADA FML, Transition plans,\n\n\nCareer and Cancer, ACS\n\n\n48\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:48}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:62}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n\n\n| Question: | n | Percent |\n| --- | --- | --- |\n| Spanish, Hispanic or Latino / Latina |  |  |\n| Yes | 3 | 7.0% |\n| No | 40 | 93.0% |\n|  |  |  |\n| Race |  |  |\n| White | 29 | 72.5% |\n| Black or African-American | 3 | 7.5% |\n| Asian | 4 | 10.0% |\n| Native Hawaiian/Pacific Islander | 0 | 0.0% |\n| American Indian/Alaskan Native | 1 | 2.5% |\n| 2 or more | 1 | 2.5% |\n| Other (specify) | 2 | 5.0% |\n|  |  |  |\n| Levels of Education |  |  |\n| No schooling | 0 | 0.0% |\n| Some high school | 0 | 0.0% |\n| Earned HS diploma/GED | 1 | 2.2% |\n| Some college | 3 | 6.7% |\n| Tech/vocational school | 0 | 0.0% |\n| Earned associates | 4 | 8.9% |\n| Earned bachelors | 16 | 35.6% |\n| At least some graduate school | 21 | 46.7% |\n|  |  |  |\n| Have you ever received treatment for any cancer? |  |  |\n| No | 36 | 80.0% |\n| Skin (melanoma) | 1 | 2.2% |\n| Skin (not melanoma) | 2 | 4.4% |\n| Yes (not skin) | 6 | 13.3% |\n\n\n62\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:62}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:84}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 8)    6. During the past 4 weeks, to what extent has your physical health or emotional problems interfered with your normal social activities with family, friends, neighbors, or groups?\n\n\nO Not at all\n\n\nO Slightly\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\n● 9)    7. How much bodily pain have you had during the past 4 weeks?\n\n\nO None\n\n\nO Very mild\n\n\nO Mild\n\n\nO Moderate\n\n\nO Severe\n\n\nO Very severe\n\n\n● 10)    8. During the past 4 weeks, how much did pain interfere with your normal work (including both work outside the home and housework)?\n\n\nO Not at all\n\n\nO A little bit\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n84\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:84}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:70}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 7)    6. During the past 4 weeks, to what extent has your physical health or emotional problems interfered with your normal social activities with family, friends, neighbors, or groups?\n\n\nO Not at all\n\n\nO Slightly\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\n● 8)    7. How much bodily pain have you had during the past 4 weeks?\n\n\nO None\n\n\nO Very mild\n\n\nO Mild\n\n\nO Moderate\n\n\nO Severe\n\n\nO Very severe\n\n\n● 9)    8. During the past 4 weeks, how much did pain interfere with your normal work (including both work outside the home and housework)?\n\n\nO Not at all\n\n\nO A little bit\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36v2® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n70\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:70}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:86}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n13) 11. How TRUE or FALSE is each of the following statements for you?\n\n\n\n\n|  | Definitely true | Mostly true | Don't know | Mostly false | Definitely false |\n| --- | --- | --- | --- | --- | --- |\n| a. I seem to get sick a little easier than other people | O | O | O | O | O |\n| b. I am as healthy as anybody I know | O | O | O | O | O |\n| c. I expect my health to get worse | O | O | O | o | o |\n| d. My health is excellent | O | O | O | o | o |\n\n\nThank you for completing these questions!\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n86\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:86}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:72}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n12) 11. How TRUE or FALSE is each of the following statements for you?\n\n\n\n\n|  | Definitely true | Mostly true | Don't know | Mostly false | Definitely false |\n| --- | --- | --- | --- | --- | --- |\n| a. I seem to get sick a little easier than other people | O | O | O | O | O |\n| b. I am as healthy as anybody I know | O | O | O | O | O |\n| c. I expect my health to get worse | O | O | O | o | o |\n| d. My health is excellent | O | O | O | o | o |\n\n\nThank you for completing these questions!\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36v2® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n72\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:72}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:100}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 25)    What is your race?\n\n\nO White\n\n\nO Black or African-American\n\n\nO Asian\n\n\nO Native Hawaiian or Other Pacific Islander\n\n\nO American Indian or Alaska Native\n\n\nO Two or more races\n\n\nO Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 26)    What is the highest grade of school you have completed?\n\n\nO I never went to school.\n\n\nO Did not complete high school.\n\n\nO Completed high school or received a high school diploma equivalent, like the GED.\n\n\nO Completed some college\n\n\nO Finished technical/vocational school\n\n\nO Finished an Associates Degree (from a 2-year college)\n\n\nO Finished a Bachelor's Degree (BA or equivalent diploma from a 4-year college)\n\n\nO Finished some or all of a graduate degree (e.g. Master's degree or Ph.D.)\n\n\n● 27)    Have you ever received treatment for any cancer?\n\n\nO No\n\n\nO Yes, skin cancer (melanoma)\n\n\nO Yes, skin cancer (not melanoma)\n\n\nO Yes, some other kind of cancer\n\n\nSECTION FIVE: CLOSING QUESTION\n\n\n● 28)    Please let us know if you have any additional comments about the eToolkit in particular, or about cancer and workplace issues in general. In your comments, please do not give us any information that would reveal the identity of anyone you know who has cancer.\n\n\nThanks so much. We really appreciate your taking the time to respond and will use the information to improve the eToolkit. Please be sure to hit \"Submit survey\" so your answers will be recorded.\n\n\n100\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:100}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:15}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nHave you disclosed your cancer (or history of cancer) to any supervisor or human resource staff member at the company that sent you the survey invitation?\n\n\nBaseline:\n\n\n\n\n| Response | n | Percent |\n| --- | --- | --- |\n| Yes | 797 | 81.6% |\n| No | 180 | 18.4% |\n\n\nFollow-up:\n\n\n\n\n| Response | n | Percent |\n| --- | --- | --- |\n| Yes | 478 | 81.3% |\n| No | 110 | 18.7% |\n\n\n15\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:15}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:8}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nTable 1. Selected baseline characteristics for all participants and for the subset with follow-up surveys\n\n\n\n\n|  | All participants (n=985) | | Participants with follow-up surveys (n=592) | |\n| --- | --- | --- | --- | --- |\n| No. | % | No. | % |\n| Sex |  |  |  |  |\n| Female | 580 | 59% | 373 | 63% |\n| Male | 400 | 41% | 218 | 37% |\n| Age |  |  |  |  |\n| 18 - 24 years | 10 | 1% | 7 | 1% |\n| 25 - 34 years | 37 | 4% | 18 | 3% |\n| 35 - 44 years | 123 | 12% | 82 | 14% |\n| 45 - 54 years | 342 | 35% | 204 | 34% |\n| 55 - 64 years | 408 | 41% | 249 | 42% |\n| 65 - 74 years | 59 | 6% | 31 | 5% |\n| 75 years or older | 1 | 0% | 1 | 0% |\n| Type of paid leave available |  |  |  |  |\n| Short/long term disability | 839 | 85% | 514 | 87% |\n| Paid time off | 842 | 85% | 524 | 89% |\n| None | 21 | 2% | 9 | 2% |\n| Other | 28 | 3% | 19 | 3% |\n| Cancer Type |  |  |  |  |\n| Bladder | 19 | 2% | 14 | 2% |\n| Breast | 314 | 32% | 221 | 37% |\n| Colon or rectum | 69 | 7% | 42 | 7% |\n| Kidney | 26 | 3% | 16 | 3% |\n| Lung | 13 | 1% | 6 | 1% |\n| Lymphoma | 73 | 7% | 42 | 7% |\n| Other | 155 | 16% | 87 | 15% |\n| Prostate | 94 | 10% | 48 | 8% |\n| Skin (melanoma) | 56 | 6% | 27 | 5% |\n| Skin (not melanoma) | 48 | 5% | 30 | 5% |\n| Testis (testicular cancer) | 25 | 3% | 17 | 3% |\n| Thyroid | 46 | 5% | 26 | 4% |\n| Uterine (endometrial cancer) | 35 | 4% | 15 | 3% |\n| Time since cancer diagnosis |  |  |  |  |\n| 6 months ago or less | 102 | 10% | 67 | 11% |\n| 7-12 months ago | 107 | 11% | 67 | 11% |\n| 13 to 24 months ago | 143 | 15% | 83 | 14% |\n| 2-5 years ago | 291 | 30% | 181 | 31% |\n| more than 5 years ago | 334 | 34% | 194 | 33% |\n| Company |  |  |  |  |\n| 1 | 110 | 11% | 64 | 11% |\n| 2 | 60 | 6% | 39 | 7% |\n| 3 | 7 | 1% | 6 | 1% |\n| 4 | 543 | 55% | 312 | 53% |\n| 5 | 45 | 5% | 25 | 4% |\n| 6 | 203 | 21% | 135 | 23% |\n| Job tenure |  |  |  |  |\n| 6 months of less | 29 | 3% | 13 | 2% |\n| 7 to 12 months | 34 | 3% | 22 | 4% |\n| 13 to 24 months | 51 | 5% | 31 | 5% |\n| 2 years or more | 866 | 88% | 523 | 88% |\n| Education |  |  |  |  |\n| Didn't finish high school | 1 | 0% | 1 | 0% |\n| Finished high school /GED | 61 | 6% | 40 | 7% |\n| Some college | 152 | 15% | 96 | 16% |\n| Finished technical/vocational school | 41 | 4% | 26 | 4% |\n| Finished BA | 312 | 32% | 177 | 30% |\n| Finished Associates | 85 | 9% | 51 | 9% |\n| Finished some or all graduate school | 331 | 34% | 200 | 34% |\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:8}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:83}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 6)    4. During the past 4 weeks, how much of the time have you had any of the following problems with your work or other regular daily activities as a result of your physical health?\n\n\n\n|  | All of the time | Most of the time | Some of the time | A little of the time | None of the time |\n| --- | --- | --- | --- | --- | --- |\n| a. Cut down on the amount of time you spent on work or other activities | O | O | O | O | O |\n| b. Accomplished less than you would like | O | O | O | O | O |\n| c. Were limited in the kind of work or other activities | O | O | O | O | O |\n| d. Had difficulty performing the work or other activities (for example, it took extra effort) | O | O | O | O | O |\n● 7)    5. During the past 4 weeks, how much of the time have you had any of the following problems with your work or other regular daily activities as a result of any emotional problems (such as feeling depressed or anxious)?\n\n\n\n|  | All of the time | Most of the time | Some of the time | A little of the time | None of the time |\n| --- | --- | --- | --- | --- | --- |\n| a. Cut down on the amount of time you spent on work or other activities | O | O | O | O | O |\n| b. Accomplished less than you would like | O | O | O | O | O |\n| c. Did work or other activities less carefully than usual | o | O | O | O | O |\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n83\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:83}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:69}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 5)    4. During the past 4 weeks, how much of the time have you had any of the following problems with your work or other regular daily activities as a result of your physical health?\n\n\n\n|  | All of the time | Most of the time | Some of the time | A little of the time | None of the time |\n| --- | --- | --- | --- | --- | --- |\n| a. Cut down on the amount of time you spent on work or other activities | O | O | O | O | O |\n| b. Accomplished less than you would like | O | O | O | O | O |\n| c. Were limited in the kind of work or other activities | O | O | O | O | O |\n| d. Had difficulty performing the work or other activities (for example, it took extra effort) | O | O | O | O | O |\n● 6)    5. During the past 4 weeks, how much of the time have you had any of the following problems with your work or other regular daily activities as a result of any emotional problems (such as feeling depressed or anxious)?\n\n\n\n|  | All of the time | Most of the time | Some of the time | A little of the time | None of the time |\n| --- | --- | --- | --- | --- | --- |\n| a. Cut down on the amount of time you spent on work or other activities | O | O | O | O | O |\n| b. Accomplished less than you would like | O | O | O | O | O |\n| c. Did work or other activities less carefully than usual | o | O | O | O | O |\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36v2® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n69\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:69}})\"]\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:82}})\"]\n\n\n===   [DOCUMENTATION_END]"
    }, {
        "role": "user",
        "content": "What is the most common cancer type?"
    }
]

Answer:

Based onfigured

SF-

SF-

SF-

SF-

SF-

<| | |

[...

Working example, with slightly less tokens in the context:

[{
        "role": "system",
        "content": "**Context:**  \nYou are an AI assistent for kohell named kohell assistant. The user is requesting content of a file.\nThe file content matching the user input is wrapped by [DOCUMENTATION_START] and [DOCUMENTATION_END].\n**Your Task:**\nAnalyze the provided content in this context thoroughly.  Answer all questions truthfully based solely on the information within this content. If you cannot find an answer within the content, state that the information is not available. Do not speculate or provide information from external sources.  \n**Formatting:**\nWhen encountering enumerations like 1, 2, 3 or tables in markdown format within the documentation, convert them into HTML for better presentation.  Use ordered lists (<ol>) for numbered enumerations and unordered lists (<ul>) for bullet points that are represented in markdown with dashes followed by a space (- ) or large comma (,) seperated lists. Use HTML tables (<table>) for presenting data in tabular format. Convert markdown syntax to html tags as needed.   \n**Please remember:**\nMaintain a neutral and objective tone throughout the conversation.  Focus on providing clear, concise answers.  You love to help your colleagues out."
    }, {
        "role": "user",
        "content": "[DOCUMENTATION_START]   \n===\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:26}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n\n\n| Cancer Type (Baseline) | n | Percent | Cancer Type (Follow-up) | n | Percent |\n| --- | --- | --- | --- | --- | --- |\n| Breast | 314 | 31.9% | Breast | 221 | 37.4% |\n| Prostate | 94 | 9.8% | Prostate | 48 | 8.1% |\n| Lymphoma | 73 | 7.6% | Lymphoma | 42 | 7.1% |\n| Colon or rectum | 69 | 7.2% | Colon or rectum | 42 | 7.1% |\n| Skin-Melanoma | 56 | 5.9% | Skin-Melanoma | 27 | 4.6% |\n| Skin-Not Melanoma | 48 | 5.0% | Skin-Not Melanoma | 30 | 5.1% |\n| Thyroid | 46 | 4.8% | Thyroid | 26 | 4.4% |\n| Uterine | 35 | 3.7% | Uterine | 15 | 2.5% |\n| Kidney | 26 | 2.7% | Kidney | 16 | 2.7% |\n| Testis | 25 | 2.6% | Testis | 17 | 2.9% |\n| Bladder | 19 | 2.0% | Bladder | 14 | 2.4% |\n| Lung | 13 | 1.4% | Lung | 6 | 1.0% |\n| Other (please specify) | 155 | 15.7% | Other (please specify) | 87 | 14.7% |\n\n\n“Other” Types of Cancer:\n\n\nHead and Neck Cancer: (n = 24)\n\n\nLeukemia: (n = 21)\n\n\nOvarian Cancer: (n = 20) Cervical cancer (n = 13) Sarcoma: (n = 12)\n\n\nTwo or more Types: (n = 9) Pancreatic Cancer: (n = 5) Brain Tumor: (n = 4) Multiple Myeloma: (n = 4) Esophageal cancer: (n = 3) Uterine Cancer: (n = 3) Lung Cancer: (n = 3) Carcinoid (n = 2)\n\n\nLiver Cancer: (n = 2) Ocular Melanoma: (n = 2) Salivary Gland Tumor: (n = 2)\n\n\nAmyloidosis: (n = 1)\n\n\nBlood Cancer: (n = 1)\n\n\nCarcinoma of unknown origin: (n = 1)\n\n\nColon Cancer: (n = 1)\n\n\nGastrointestinal stromal tumor: (n = 1)\n\n\nGestational trophoblastic disease: (n = 1)\n\n\nNasal/Brain Tumor: (n = 1)\n\n\nNeuroblastoma: (n = 1)\n\n\nParaganglioma: (n = 1)\n\n\nPrimary Peritoneal Cancer: (n = 1)\n\n\nSpinal Ependymoma: (n = 1)\n\n\nThymus Cancer: (n = 1)\n\n\nThyroid Cancer: (n = 1)\n\n\nVaginal Cancer: (n = 1)\n\n\nVulvar Cancer: (n = 1)\n\n\n26\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:26}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:30}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nEmployee Survey - Follow-up Data\n\n\nSince July 2014, have you been diagnosed with a kind of cancer that is different from the diagnosis you had then?\n\n\n\n\n| Response - Yes | n | Percent |\n| --- | --- | --- |\n| Skin (not melanoma) | 3 | 0.5% |\n| Lymphoma | 1 | 0.2% |\n| Prostate | 1 | 0.2% |\n| Skin (melanoma) | 1 | 0.2% |\n| Kidney | 1 | 0.2% |\n| Other (please specify) | 6 | 1.0% |\n| Response - No | 579 | 97.8% |\n\n\nOther Reponses:\n\n\nCervical (n = 2)\n\n\nPyriform Sinus/Laryngeal Cancer (n = 1)\n\n\nNeuroendocrine cancer (pancreas liver cancer) (n = 1)\n\n\nStomach (n = 1)\n\n\nOvarian (n = 1)\n\n\n30\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:30}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:89}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 24)    Which kind of cancer diagnosis have you received since July 2014 in addition to the one you had then?\n\n\nO Bladder\n\n\nO Breast\n\n\nO Colon or rectum\n\n\nO Kidney\n\n\nO Lung\n\n\nO Lymphoma\n\n\nO Prostate\n\n\nO Skin (melanoma)\n\n\nO Skin (not melanoma)\n\n\nO Testis (testicular cancer)\n\n\nO Thyroid\n\n\nO Uterine (also known as endometrial cancer)\n\n\nO Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 25)    Since July 2014, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n● □    I have not had any cancer treatment in the last 6 months.\n● □    Oral medication (EXCEPT oral chemotherapy)\n● □    Radiation\n● □    Surgery\n● □    Oral chemotherapy\n● □    Intravenous chemotherapy (may be given through your vein or a through a catheter or mediport)\n● □    Other chemotherapy (This may include chemotherapy given directly into the spinal fluid or into the peritoneal cavity. Note that these kinds of chemotherapy are very rare.)\n● □    Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 26)    Have you disclosed your cancer (or history of cancer) to any supervisor or human resource staff member at the company that sent you the invitation to the first survey?\n\n\nO Yes\n\n\nO No [Skips to question 28]\n\n\n89\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:89}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:28}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nDuring the last 6 months, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n\n\n| Response (Baseline) | n | Percent |\n| --- | --- | --- |\n| I have not had any cancer treatment in the last 6 months. | 566 | 57.5% |\n| Surgery | 168 | 17.1% |\n| Oral medication (EXCEPT oral chemotherapy) | 147 | 14.9% |\n| Radiation | 83 | 8.4% |\n| Intravenous chemotherapy (may be given through your vein or a through a catheter or mediport) | 75 | 7.6% |\n| Oral chemotherapy | 43 | 4.4% |\n| Other chemotherapy (This may include chemotherapy given directly into the spinal fluid or into the peritoneal cavity. Note that these kinds of chemotherapy are very rare.) | 3 | 0.3% |\n| Other (please specify) | 91 | 9.2% |\n\n\nExamples of “Other” kinds of treatment:\n\n\nInjected Medicines (not chemotherapy): (n = 22)\n\n\nScans and/or Imaging Services (full body scan, mammogram, ultrasound, cystoscopy, MRI,\n\n\nCAT Scan, sonogram): (n = 16)\n\n\nFollow-up Visits: (n = 12)\n\n\nSkin Treatments (freezing/burning/biopsies/grafts): (n = 10)\n\n\nBlood Tests: (n = 5)\n\n\nHormone therapy: (n = 4)\n\n\nPhysical Therapy: (n = 4)\n\n\nStem Cell Transplant: (n = 2)\n\n\nDiagnostic Testing: (n = 2)\n\n\nTopical Chemotherapy: (n = 2)\n\n\nInhaler: (n = 1)\n\n\nRadioactive Iodine: (n = 1)\n\n\nParticipant in a Blind Study: (n = 1)\n\n\n28\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:28}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:75}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 23)    How long ago did you receive this cancer diagnosis?\n\n\nO 6 months ago or less\n\n\nO 7 to 12 months ago\n\n\nO 13 to 24 months ago\n\n\nO 2 to 5 years ago\n\n\nO More than 5 years ago\n\n\n● 24)    During the last 6 months, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n● □    I have not had any cancer treatment in the last 6 months.\n● □    Oral medication (EXCEPT oral chemotherapy)\n\n\n□ Radiation\n\n\n● □    Surgery\n● □    Oral chemotherapy\n● □    Intravenous chemotherapy (may be given through your vein or a through a catheter or mediport)\n● □    Other chemotherapy (This may include chemotherapy given directly into the spinal fluid or into the peritoneal cavity. Note that these kinds of chemotherapy are very rare.)\n● □    Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 25)    Have you disclosed your cancer (or history of cancer) to any supervisor or human resource staff member at the company that sent you the survey invitation?\n\n\nO Yes\n\n\nO No [Skip to question 27]\n\n\n75\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:75}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:31}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nSince July 2014, which of the following kinds of treatment have you had for your cancer? (Select all that apply.)\n\n\n\n\n| Response (Follow-up) | n | Percent |\n| --- | --- | --- |\n| None | 374 | 54.8% |\n| Oral (not oral chemo) | 110 | 16.1% |\n| Radiation | 33 | 4.8% |\n| Surgery | 48 | 7.0% |\n| Oral chemotherapy | 22 | 3.2% |\n| IV chemo | 41 | 6.0% |\n| Other chemo | 1 | 0.1% |\n| Other (please specify) | 53 | 7.8% |\n\n\nOther Responses:\n\n\nInjected Medicines (not chemotherapy): (n = 12)\n\n\nFollow-up Visits: (n = 12)\n\n\nScans and/or Imaging Services (full body scan, mammogram, ultrasound, cystoscopy, MRI,\n\n\nCAT Scan, sonogram): (n = 8)\n\n\nSkin Treatments (freezing/burning/biopsies/grafts): (n = 8)\n\n\nPhysical Therapy: (n = 3)\n\n\nBlood Tests: (n = 2)\n\n\nHormone therapy: (n = 2)\n\n\nDiagnostic Testing: (n = 2)\n\n\nTopical Chemotherapy: (n = 2)\n\n\nIncontinence devices: (n = 1)\n\n\nFluoride Treatment: (n = 1)\n\n\n31\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:31}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:27}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nEmployee Survey Report - Baseline Data\n\n\nHow long ago did you receive this cancer diagnosis?\n\n\n\n\n| Response (Baseline) | n | Percent |\n| --- | --- | --- |\n| < 6 mo. | 102 | 10.4% |\n| 7-12 mo. | 107 | 11.0% |\n| 13-24 mo. | 143 | 14.6% |\n| 2-5 yrs. | 291 | 29.8% |\n| > 5 yrs. | 334 | 34.2% |\n\n\n27\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:27}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:48}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nThe eToolkit often suggested other sources of information about cancer and workplace issues. Since receiving access to the eToolkit, have you looked up any of those resources?\n\n\nWhat were some of those resources?\n\n\nAll of them.\n\n\nMy brother has Leukemia and my father was diagnosed yesterday as terminal with another type of blood cancer. I was just reviewing the site and pleased to see the information that exists for those impacted by these horrible diseases.\n\n\nCoping with the effects of Chemo and Radiation Chemo-Brain\n\n\nCaregiver, American Cancer Society\n\n\nI have lost 3 family members in the last 18 months\n\n\nThe links to cancer information regarding side effects of treatment, etc. Can't remember the link.\n\n\nProton Radiation program at Loma Linda Medical Center\n\n\nADA FML, Transition plans,\n\n\nCareer and Cancer, ACS\n\n\n48\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:48}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:62}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n\n\n| Question: | n | Percent |\n| --- | --- | --- |\n| Spanish, Hispanic or Latino / Latina |  |  |\n| Yes | 3 | 7.0% |\n| No | 40 | 93.0% |\n|  |  |  |\n| Race |  |  |\n| White | 29 | 72.5% |\n| Black or African-American | 3 | 7.5% |\n| Asian | 4 | 10.0% |\n| Native Hawaiian/Pacific Islander | 0 | 0.0% |\n| American Indian/Alaskan Native | 1 | 2.5% |\n| 2 or more | 1 | 2.5% |\n| Other (specify) | 2 | 5.0% |\n|  |  |  |\n| Levels of Education |  |  |\n| No schooling | 0 | 0.0% |\n| Some high school | 0 | 0.0% |\n| Earned HS diploma/GED | 1 | 2.2% |\n| Some college | 3 | 6.7% |\n| Tech/vocational school | 0 | 0.0% |\n| Earned associates | 4 | 8.9% |\n| Earned bachelors | 16 | 35.6% |\n| At least some graduate school | 21 | 46.7% |\n|  |  |  |\n| Have you ever received treatment for any cancer? |  |  |\n| No | 36 | 80.0% |\n| Skin (melanoma) | 1 | 2.2% |\n| Skin (not melanoma) | 2 | 4.4% |\n| Yes (not skin) | 6 | 13.3% |\n\n\n62\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:62}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:84}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 8)    6. During the past 4 weeks, to what extent has your physical health or emotional problems interfered with your normal social activities with family, friends, neighbors, or groups?\n\n\nO Not at all\n\n\nO Slightly\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\n● 9)    7. How much bodily pain have you had during the past 4 weeks?\n\n\nO None\n\n\nO Very mild\n\n\nO Mild\n\n\nO Moderate\n\n\nO Severe\n\n\nO Very severe\n\n\n● 10)    8. During the past 4 weeks, how much did pain interfere with your normal work (including both work outside the home and housework)?\n\n\nO Not at all\n\n\nO A little bit\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n84\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:84}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:70}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 7)    6. During the past 4 weeks, to what extent has your physical health or emotional problems interfered with your normal social activities with family, friends, neighbors, or groups?\n\n\nO Not at all\n\n\nO Slightly\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\n● 8)    7. How much bodily pain have you had during the past 4 weeks?\n\n\nO None\n\n\nO Very mild\n\n\nO Mild\n\n\nO Moderate\n\n\nO Severe\n\n\nO Very severe\n\n\n● 9)    8. During the past 4 weeks, how much did pain interfere with your normal work (including both work outside the home and housework)?\n\n\nO Not at all\n\n\nO A little bit\n\n\nO Moderately\n\n\nO Quite a bit\n\n\nO Extremely\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36v2® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n70\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:70}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:86}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n13) 11. How TRUE or FALSE is each of the following statements for you?\n\n\n\n\n|  | Definitely true | Mostly true | Don't know | Mostly false | Definitely false |\n| --- | --- | --- | --- | --- | --- |\n| a. I seem to get sick a little easier than other people | O | O | O | O | O |\n| b. I am as healthy as anybody I know | O | O | O | O | O |\n| c. I expect my health to get worse | O | O | O | o | o |\n| d. My health is excellent | O | O | O | o | o |\n\n\nThank you for completing these questions!\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n86\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:86}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:72}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n12) 11. How TRUE or FALSE is each of the following statements for you?\n\n\n\n\n|  | Definitely true | Mostly true | Don't know | Mostly false | Definitely false |\n| --- | --- | --- | --- | --- | --- |\n| a. I seem to get sick a little easier than other people | O | O | O | O | O |\n| b. I am as healthy as anybody I know | O | O | O | O | O |\n| c. I expect my health to get worse | O | O | O | o | o |\n| d. My health is excellent | O | O | O | o | o |\n\n\nThank you for completing these questions!\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36v2® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n72\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:72}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:100}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 25)    What is your race?\n\n\nO White\n\n\nO Black or African-American\n\n\nO Asian\n\n\nO Native Hawaiian or Other Pacific Islander\n\n\nO American Indian or Alaska Native\n\n\nO Two or more races\n\n\nO Other (please specify)\n\n\nIf you selected other, please specify\n\n\n● 26)    What is the highest grade of school you have completed?\n\n\nO I never went to school.\n\n\nO Did not complete high school.\n\n\nO Completed high school or received a high school diploma equivalent, like the GED.\n\n\nO Completed some college\n\n\nO Finished technical/vocational school\n\n\nO Finished an Associates Degree (from a 2-year college)\n\n\nO Finished a Bachelor's Degree (BA or equivalent diploma from a 4-year college)\n\n\nO Finished some or all of a graduate degree (e.g. Master's degree or Ph.D.)\n\n\n● 27)    Have you ever received treatment for any cancer?\n\n\nO No\n\n\nO Yes, skin cancer (melanoma)\n\n\nO Yes, skin cancer (not melanoma)\n\n\nO Yes, some other kind of cancer\n\n\nSECTION FIVE: CLOSING QUESTION\n\n\n● 28)    Please let us know if you have any additional comments about the eToolkit in particular, or about cancer and workplace issues in general. In your comments, please do not give us any information that would reveal the identity of anyone you know who has cancer.\n\n\nThanks so much. We really appreciate your taking the time to respond and will use the information to improve the eToolkit. Please be sure to hit \"Submit survey\" so your answers will be recorded.\n\n\n100\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:100}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:15}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nHave you disclosed your cancer (or history of cancer) to any supervisor or human resource staff member at the company that sent you the survey invitation?\n\n\nBaseline:\n\n\n\n\n| Response | n | Percent |\n| --- | --- | --- |\n| Yes | 797 | 81.6% |\n| No | 180 | 18.4% |\n\n\nFollow-up:\n\n\n\n\n| Response | n | Percent |\n| --- | --- | --- |\n| Yes | 478 | 81.3% |\n| No | 110 | 18.7% |\n\n\n15\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:15}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:8}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\nTable 1. Selected baseline characteristics for all participants and for the subset with follow-up surveys\n\n\n\n\n|  | All participants (n=985) | | Participants with follow-up surveys (n=592) | |\n| --- | --- | --- | --- | --- |\n| No. | % | No. | % |\n| Sex |  |  |  |  |\n| Female | 580 | 59% | 373 | 63% |\n| Male | 400 | 41% | 218 | 37% |\n| Age |  |  |  |  |\n| 18 - 24 years | 10 | 1% | 7 | 1% |\n| 25 - 34 years | 37 | 4% | 18 | 3% |\n| 35 - 44 years | 123 | 12% | 82 | 14% |\n| 45 - 54 years | 342 | 35% | 204 | 34% |\n| 55 - 64 years | 408 | 41% | 249 | 42% |\n| 65 - 74 years | 59 | 6% | 31 | 5% |\n| 75 years or older | 1 | 0% | 1 | 0% |\n| Type of paid leave available |  |  |  |  |\n| Short/long term disability | 839 | 85% | 514 | 87% |\n| Paid time off | 842 | 85% | 524 | 89% |\n| None | 21 | 2% | 9 | 2% |\n| Other | 28 | 3% | 19 | 3% |\n| Cancer Type |  |  |  |  |\n| Bladder | 19 | 2% | 14 | 2% |\n| Breast | 314 | 32% | 221 | 37% |\n| Colon or rectum | 69 | 7% | 42 | 7% |\n| Kidney | 26 | 3% | 16 | 3% |\n| Lung | 13 | 1% | 6 | 1% |\n| Lymphoma | 73 | 7% | 42 | 7% |\n| Other | 155 | 16% | 87 | 15% |\n| Prostate | 94 | 10% | 48 | 8% |\n| Skin (melanoma) | 56 | 6% | 27 | 5% |\n| Skin (not melanoma) | 48 | 5% | 30 | 5% |\n| Testis (testicular cancer) | 25 | 3% | 17 | 3% |\n| Thyroid | 46 | 5% | 26 | 4% |\n| Uterine (endometrial cancer) | 35 | 4% | 15 | 3% |\n| Time since cancer diagnosis |  |  |  |  |\n| 6 months ago or less | 102 | 10% | 67 | 11% |\n| 7-12 months ago | 107 | 11% | 67 | 11% |\n| 13 to 24 months ago | 143 | 15% | 83 | 14% |\n| 2-5 years ago | 291 | 30% | 181 | 31% |\n| more than 5 years ago | 334 | 34% | 194 | 33% |\n| Company |  |  |  |  |\n| 1 | 110 | 11% | 64 | 11% |\n| 2 | 60 | 6% | 39 | 7% |\n| 3 | 7 | 1% | 6 | 1% |\n| 4 | 543 | 55% | 312 | 53% |\n| 5 | 45 | 5% | 25 | 4% |\n| 6 | 203 | 21% | 135 | 23% |\n| Job tenure |  |  |  |  |\n| 6 months of less | 29 | 3% | 13 | 2% |\n| 7 to 12 months | 34 | 3% | 22 | 4% |\n| 13 to 24 months | 51 | 5% | 31 | 5% |\n| 2 years or more | 866 | 88% | 523 | 88% |\n| Education |  |  |  |  |\n| Didn't finish high school | 1 | 0% | 1 | 0% |\n| Finished high school /GED | 61 | 6% | 40 | 7% |\n| Some college | 152 | 15% | 96 | 16% |\n| Finished technical/vocational school | 41 | 4% | 26 | 4% |\n| Finished BA | 312 | 32% | 177 | 30% |\n| Finished Associates | 85 | 9% | 51 | 9% |\n| Finished some or all graduate school | 331 | 34% | 200 | 34% |\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:8}})\"]\n\n[SECTION_START: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:83}})\"]\n\n\n\nWorkplaceTransitionsPilotStudy-TechnicalReport-Final\n\n\nWorkplace Transitions for People Touched by Cancer Pilot Study Report, July 2015\n\n\n● 6)    4. During the past 4 weeks, how much of the time have you had any of the following problems with your work or other regular daily activities as a result of your physical health?\n\n\n\n|  | All of the time | Most of the time | Some of the time | A little of the time | None of the time |\n| --- | --- | --- | --- | --- | --- |\n| a. Cut down on the amount of time you spent on work or other activities | O | O | O | O | O |\n| b. Accomplished less than you would like | O | O | O | O | O |\n| c. Were limited in the kind of work or other activities | O | O | O | O | O |\n| d. Had difficulty performing the work or other activities (for example, it took extra effort) | O | O | O | O | O |\n● 7)    5. During the past 4 weeks, how much of the time have you had any of the following problems with your work or other regular daily activities as a result of any emotional problems (such as feeling depressed or anxious)?\n\n\n\n|  | All of the time | Most of the time | Some of the time | A little of the time | None of the time |\n| --- | --- | --- | --- | --- | --- |\n| a. Cut down on the amount of time you spent on work or other activities | O | O | O | O | O |\n| b. Accomplished less than you would like | O | O | O | O | O |\n| c. Did work or other activities less carefully than usual | o | O | O | O | O |\n\n\nSF-36v2® Health Survey © 1992, 1996, 2000 Medical Outcomes Trust and QualityMetric Incorporated. All rights reserved.\n\n\nSF-36® is a registered trademark of Medical Outcomes Trust.\n\n\n(SF-36v2® Health Survey Standard, United States (English))\n\n\n83\n\n\n\n\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:83}})\"]\n\n[SECTION_END: \"[Documentation]({{file:735e7def-2521-488c-ae41-5cb9c0f843b6:page:69}})\"]\n\n\n===   [DOCUMENTATION_END]"
    }, {
        "role": "user",
        "content": "What is the most common cancer type?"
    }
]

Answer:

Based on the provided content, the most common cancer type at baseline is Breast cancer, with 314 participants (31.9%) having this type of cancer.

nFunctor commented 1 month ago

If I understand correctly passing --quantization “awq” as quantisation argument for the server should deactivate the Marlin kernels, maybe this could determine if they are a nonetheless a cause in a particular pipeline?

Personally I’ve run into some odd repetitions when using AWQ models with flashinfer backend during old vs new beam search tests. That attention backend even seemed to impact the generation for prompts[0] depending on the rest of the batch size… thought to share these details since the issue seems unclear.

ywang96 commented 2 weeks ago

Closing as fixed by https://github.com/vllm-project/vllm/pull/9549