Add scripted load tests

[stuartleeks]

• Initial work to automate running load tests against a deployed simulator instance
• Load tests
  • Load test: no limits and no added latency - to test baseline performance
  • Load test: no limits and 1s latency - to test simulated latency is correct
  • Load test: 10 requests per second limit - to test that the RPS limiting is correct
  • Load test: 100,000 tokens per minute limit - to test that the TPM limiting is correct
• Improve lorem ipsum text generation performance
  • use cached token strings and assemble response content from these to avoid repeated calls to lorem generation token counting
  • Fix: include streaming responses in token counts
  • Update metric names for token metrics to match latency naming
  • Split out prompt vs completion tokens using metric dimensions
• Replace python-lorem with custom implementation to resolve build errors (see below)
• Add docs page for simulator metrics

Closes #6

Build output for failed python-lorem installation

[72 lines of output] /tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/dist.py:478: SetuptoolsDeprecationWarning: Invalid dash-separated options !! ******************************************************************************** Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead. By 2024-Sep-26, you need to update your project and remove deprecated calls or your builds will no longer be supported. See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details. ******************************************************************************** !! opt = self.warn_dash_deprecation(opt, section) running dist_info creating /tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info writing /tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info/PKG-INFO writing dependency_links to /tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info/dependency_links.txt writing requirements to /tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info/requires.txt writing top-level names to /tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info/top_level.txt writing manifest file '/tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no previously-included files matching '.DS_Store' found anywhere in distribution warning: no files found matching '*.rst' anywhere in distribution no previously-included directories found matching '.eggs' no previously-included directories found matching '.github' no previously-included directories found matching '.venv' no previously-included directories found matching 'docs' no previously-included directories found matching 'temp' adding license file 'LICENSE' writing manifest file '/tmp/pip-modern-metadata-uo9lo7si/python_lorem.egg-info/SOURCES.txt' creating '/tmp/pip-modern-metadata-uo9lo7si/python_lorem-1.3.0.post1.dist-info' Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 368, in prepare_metadata_for_build_wheel self.run_setup() File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup exec(code, locals()) File "", line 200, in File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup return distutils.core.setup(**attrs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup return run_commands(dist) ^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands dist.run_commands() File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command super().run_command(command) File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/command/dist_info.py", line 97, in run bdist_wheel = self.get_finalized_command('bdist_wheel') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 302, in get_finalized_command cmd_obj = self.distribution.get_command_obj(command, create) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-0z2zwjj3/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 862, in get_command_obj cmd_obj = self.command_obj[command] = klass(self) ^^^^^^^^^^^ TypeError: 'NoneType' object is not callable [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. [2024-06-21T14:34:41.442Z] error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. ```