search

Nutlope / aicommits

A CLI that writes your git commit messages for you with AI
https://www.npmjs.com/package/aicommits
MIT License
7.95k stars 381 forks source link

400 - Bad request - Maximum content length. #203

Closed sunscreem closed 1 year ago

sunscreem commented 1 year ago

Bug description

When running the command I get the following:

◇  Detected 2 staged files:
     tests/Fixtures/TicketSource/events.index.json
     tests/Unit/TicketSourceAPITest.php
│
◇  Changes analyzed
│
└  ✖ OpenAI API Error: 400 - Bad Request

{
  "error": {
    "message": "This model's maximum context length is 4097 tokens. However, you requested 5838 tokens (2909 in the messages, 2929 in the completion). Please reduce the length of the messages or completion.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}

My changes aren't particularly big. I've create a text file and added a single new test to my test suite.

git diff --cached
diff --git a/tests/Fixtures/TicketSource/events.index.json b/tests/Fixtures/TicketSource/events.index.json
index 0434c49..9d4b7e7 100644
--- a/tests/Fixtures/TicketSource/events.index.json
+++ b/tests/Fixtures/TicketSource/events.index.json
@@ -1 +1 @@
-{"statusCode":403,"headers":{"Date":"Sun, 30 Apr 2023 14:36:54 GMT","Content-Type":"text\/html; charset=UTF-8","Transfer-Encoding":"chunked","Connection":"keep-alive","Cache-Control":"max-age=15","Expires":"Sun, 30 Apr 2023 14:37:09 GMT","X-Frame-Options":"SAMEORIGIN","Report-To":"{\"endpoints\":[{\"url\":\"https:\\\/\\\/a.nel.cloudflare.com\\\/report\\\/v3?s=Vw41yz6s0hMgWrkbJWmSYwVZFcOoT5bKhg4TVpDimLV6VSBiGHbvdlD6K3Tmn4MsBLoimlt275%2FXaZUYlOrsKNGJdvlLuJ5T8qPlcXMmDD713bOkB92QDZRCoV2cHXTkdQRKCwFs\"}],\"group\":\"cf-nel\",\"max_age\":604800}","NEL":"{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}","Strict-Transport-Security":"max-age=31536000; includeSubDomains; preload","X-Content-Type-Options":"nosniff","Server":"cloudflare","CF-RAY":"7c008508f889dcdb-LHR","alt-svc":"h3=\":443\"; ma=86400, h3-29=\":443\"; ma=86400"},"data":"<!DOCTYPE html>\n<!--[if lt IE 7]> <html class=\"no-js ie6 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 7]>    <html class=\"no-js ie7 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 8]>    <html class=\"no-js ie8 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if gt IE 8]><!--> <html class=\"no-js\" lang=\"en-US\"> <!--<![endif]-->\n<head>\n<title>Attention Required! | Cloudflare<\/title>\n<meta charset=\"UTF-8\" \/>\n<meta http-equiv=\"Content-Type\" content=\"text\/html; charset=UTF-8\" \/>\n<meta http-equiv=\"X-UA-Compatible\" content=\"IE=Edge\" \/>\n<meta name=\"robots\" content=\"noindex, nofollow\" \/>\n<meta name=\"viewport\" content=\"width=device-width,initial-scale=1\" \/>\n<link rel=\"stylesheet\" id=\"cf_styles-css\" href=\"\/cdn-cgi\/styles\/cf.errors.css\" \/>\n<!--[if lt IE 9]><link rel=\"stylesheet\" id='cf_styles-ie-css' href=\"\/cdn-cgi\/styles\/cf.errors.ie.css\" \/><![endif]-->\n<style>body{margin:0;padding:0}<\/style>\n\n\n<!--[if gte IE 10]><!-->\n<script>\n  if (!navigator.cookieEnabled) {\n    window.addEventListener('DOMContentLoaded', function () {\n      var cookieEl = document.getElementById('cookie-alert');\n      cookieEl.style.display = 'block';\n    })\n  }\n<\/script>\n<!--<![endif]-->\n\n\n<\/head>\n<body>\n  <div id=\"cf-wrapper\">\n    <div class=\"cf-alert cf-alert-error cf-cookie-error\" id=\"cookie-alert\" data-translate=\"enable_cookies\">Please enable cookies.<\/div>\n    <div id=\"cf-error-details\" class=\"cf-error-details-wrapper\">\n      <div class=\"cf-wrapper cf-header cf-error-overview\">\n        <h1 data-translate=\"block_headline\">Sorry, you have been blocked<\/h1>\n        <h2 class=\"cf-subheadline\"><span data-translate=\"unable_to_access\">You are unable to access<\/span> ticketsource.io<\/h2>\n      <\/div><!-- \/.header -->\n\n      <div class=\"cf-section cf-highlight\">\n        <div class=\"cf-wrapper\">\n          <div class=\"cf-screenshot-container cf-screenshot-full\">\n            \n              <span class=\"cf-no-screenshot error\"><\/span>\n            \n          <\/div>\n        <\/div>\n      <\/div><!-- \/.captcha-container -->\n\n      <div class=\"cf-section cf-wrapper\">\n        <div class=\"cf-columns two\">\n          <div class=\"cf-column\">\n            <h2 data-translate=\"blocked_why_headline\">
Why have I been blocked?<\/h2>\n\n            <p data-translate=\"blocked_why_detail\">This website is usi
ng a security service to protect itself from online attacks. The action you just performed triggered the s
ecurity solution. There are several actions that could trigger this block including submitting a certain w
ord or phrase, a SQL command or malformed data.<\/p>\n          <\/div>\n\n          <div class=\"cf-colum
n\">\n            <h2 data-translate=\"blocked_resolve_headline\">What can I do to resolve this?<\/h2>\n\n
            <p data-translate=\"blocked_resolve_detail\">You can email the site owner to let them know you
 were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found a
t the bottom of this page.<\/p>\n          <\/div>\n        <\/div>\n      <\/div><!-- \/.section -->\n\n 
     <div class=\"cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:
text-left border-solid border-0 border-t border-gray-300\">\n  <p class=\"text-13\">\n    <span class=\"cf
-footer-item sm:block sm:mb-1\">Cloudflare Ray ID: <strong class=\"font-semibold\">7c008508f889dcdb<\/stro
ng><\/span>\n    <span class=\"cf-footer-separator sm:hidden\">&bull;<\/span>\n    <span id=\"cf-footer-it
em-ip\" class=\"cf-footer-item hidden sm:block sm:mb-1\">\n      Your IP:\n      <button type=\"button\" i
d=\"cf-footer-ip-reveal\" class=\"cf-footer-ip-reveal-btn\">Click to reveal<\/button>\n      <span class=\
"hidden\" id=\"cf-footer-ip\">167.99.206.90<\/span>\n      <span class=\"cf-footer-separator sm:hidden\">&
bull;<\/span>\n    <\/span>\n    <span class=\"cf-footer-item sm:block sm:mb-1\"><span>Performance &amp; s
ecurity by<\/span> <a rel=\"noopener noreferrer\" href=\"https:\/\/www.cloudflare.com\/5xx-error-landing\"
 id=\"brand_link\" target=\"_blank\">Cloudflare<\/a><\/span>\n    \n  <\/p>\n  <script>(function(){functio
n d(){var b=a.getElementById(\"cf-footer-item-ip\"),c=a.getElementById(\"cf-footer-ip-reveal\");b&&\"class
List\"in b&&(b.classList.remove(\"hidden\"),c.addEventListener(\"click\",function(){c.classList.add(\"hidd
en\");a.getElementById(\"cf-footer-ip\").classList.remove(\"hidden\")}))}var a=document;document.addEventL
istener&&a.addEventListener(\"DOMContentLoaded\",d)})();<\/script>\n<\/div><!-- \/.error-footer -->\n\n\n 
   <\/div><!-- \/#cf-error-details -->\n  <\/div><!-- \/#cf-wrapper -->\n\n  <script>\n  window._cf_transl
ation = {};\n  \n  \n<\/script>\n\n<\/body>\n<\/html>\n"}
\ No newline at end of file
+{"statusCode":200,"headers":{"Date":"Sun, 30 Apr 2023 14:36:54 GMT","Content-Type":"text\/html; charset=U
TF-8","Transfer-Encoding":"chunked","Connection":"keep-alive","Cache-Control":"max-age=15","Expires":"Sun,
 30 Apr 2023 14:37:09 GMT","X-Frame-Options":"SAMEORIGIN","Report-To":"{\"endpoints\":[{\"url\":\"https:\\
\/\\\/a.nel.cloudflare.com\\\/report\\\/v3?s=Vw41yz6s0hMgWrkbJWmSYwVZFcOoT5bKhg4TVpDimLV6VSBiGHbvdlD6K3Tmn
4MsBLoimlt275%2FXaZUYlOrsKNGJdvlLuJ5T8qPlcXMmDD713bOkB92QDZRCoV2cHXTkdQRKCwFs\"}],\"group\":\"cf-nel\",\"m
ax_age\":604800}","NEL":"{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}","Strict-Tra
nsport-Security":"max-age=31536000; includeSubDomains; preload","X-Content-Type-Options":"nosniff","Server
":"cloudflare","CF-RAY":"7c008508f889dcdb-LHR","alt-svc":"h3=\":443\"; ma=86400, h3-29=\":443\"; ma=86400"
},"data":"{\"data\":[{\"id\":\"evt-WE39wVbREgJVrvZY\",\"type\":\"event\",\"attributes\":{\"name\":\"Test E
vent 2\",\"reference\":\"\",\"description\":\"Lorem Ipsum\",\"comment\":\"\",\"keywords\":null,\"terms\":\
"\",\"category\":\"business\",\"genre\":\"dinner\",\"images\":[],\"third_party_consent\":{\"capture\":fals
e,\"name\":\"\",\"show_consent\":{\"email\":false,\"post\":false,\"sms\":false}},\"activated\":true,\"arch
ived\":false,\"public\":true,\"settled\":false,\"created_at\":\"2023-04-28T17:50:46+00:00\",\"updated_at\"
:\"2023-04-28T17:51:45+00:00\"},\"links\":{\"self\":\"https:\\/\\/api.ticketsource.io\\/events\\/evt-WE39w
VbREgJVrvZY\",\"venues\":\"https:\\/\\/api.ticketsource.io\\/events\\/evt-WE39wVbREgJVrvZY\\/venues\",\"da
tes\":\"https:\\/\\/api.ticketsource.io\\/events\\/evt-WE39wVbREgJVrvZY\\/dates\"}}],\"links\":{\"first\":
\"https:\\/\\/api.ticketsource.io\\/events?page=1\",\"last\":null,\"prev\":null,\"next\":null},\"meta\":{\
"current_page\":1,\"from\":1,\"path\":\"https:\\/\\/api.ticketsource.io\\/events\",\"per_page\":10,\"to\":
1}}"}
\ No newline at end of file
diff --git a/tests/Unit/TicketSourceAPITest.php b/tests/Unit/TicketSourceAPITest.php
index a58e36d..466d73d 100644
--- a/tests/Unit/TicketSourceAPITest.php
+++ b/tests/Unit/TicketSourceAPITest.php
@@ -6,6 +6,13 @@ use Saloon\Helpers\MockConfig;
 use Saloon\Http\Faking\MockResponse;
 use Saloon\Laravel\Saloon;

+beforeEach(function () {
+    MockConfig::throwOnMissingFixtures();
+
+    MockConfig::setFixturePath('tests/Fixtures/TicketSource');
+
+});
+
 it('throws_errors', function () {

     Saloon::fake([
@@ -16,28 +23,29 @@ it('throws_errors', function () {
     $ticketSource->send(new GetEvents);

     Saloon::assertSent(GetEvents::class);
-})->throws(Exception::class)->skip();
+})->throws(Exception::class);

 it('fetches_a_list_of_events', function () {

-    // rob you are here.
-    // you want to fake the real response
-    // and work out how to make the correct calls
-
-    // MockConfig::throwOnMissingFixtures();
-
-    MockConfig::setFixturePath('tests/Fixtures/TicketSource');
-
     Saloon::fake([
         TicketSourceConnector::class => MockResponse::fixture('events.index'),
     ]);

-    $ticketSource = new TicketSourceConnector;
-    $response = $ticketSource->send(new GetEvents);
+    $data = (new TicketSourceConnector)->send(new GetEvents)->json();

-    Saloon::assertSent(GetEvents::class);
+    expect($data)->toHaveCount(3);
+
+    $events = $data['data'];
+
+    expect($events)->toHaveCount(1);
+
+    expect($events[0])->toHaveKeys([
+        'id',
+        'type',
+        'attributes',
+        'links',
+    ]);

-    expect($response->status())->toBe(200);
 });

aicommits version

1.10.0

Environment

System:
    OS: Linux 4.15 Ubuntu 18.04.6 LTS (Bionic Beaver)
    CPU: (4) x64 DO-Regular
    Memory: 3.75 GB / 7.79 GB
    Container: Yes
    Shell: 4.4.20 - /bin/bash
  Binaries:
    Node: 16.16.0 - /usr/local/bin/node
    Yarn: 1.19.1 - /usr/bin/yarn
    npm: 8.11.0 - /usr/local/bin/npm

Can you contribute a fix?

sunscreem commented 1 year ago

I'm wondering if I set something up wrong as that diff doesn't seem especially large?

privatenumber commented 1 year ago

@rocktimsaikia Wondering if we need to set a max here depending on the model?

https://github.com/Nutlope/aicommits/blob/f53fb85dd75513d05fa6f1e73890684ff825e5e2/src/utils/openai.ts#L166

I think we can get the max for each model from tiktoken.

RyanRoberts commented 1 year ago

Exact same issue here. It was working fine last night with larger diffs but this morning I've done some minimal work and run into this error. Strange.

mreduar commented 1 year ago

Same here

privatenumber commented 1 year ago

I haven't had time to investigate this yet, but if anyone wants to, feel free to open a PR. I think it's basically setting a limit here: https://github.com/Nutlope/aicommits/blob/f53fb85dd75513d05fa6f1e73890684ff825e5e2/src/utils/openai.ts#L166

sunscreem commented 1 year ago

@privatenumber Sorry to report I've updated to 1.11.0 and I've still having the same issue.

npm list -g
npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
/usr/local/lib
├── aicommits@1.11.0
├── corepack@0.10.0
├── eslint@7.14.0
├── n@9.0.0
├── npm-why@3.0.0
├── npm@8.11.0
├── prettier@2.7.1
├── webpack-cli@4.5.0
└── webpack@5.24.2
  aicommits 
│
◇  Detected 13 staged files:
     app/Http/Integrations/TicketSource/Requests/GetDate.php
     app/Http/Integrations/TicketSource/Requests/GetEvent.php
     app/Http/Integrations/TicketSource/Requests/GetEventDates.php
     app/Http/Integrations/TicketSource/Requests/GetEventVenues.php
     app/Http/Integrations/TicketSource/Requests/GetVenue.php
     app/Http/Integrations/TicketSource/Requests/GetVenueDates.php
     tests/Fixtures/TicketSource/date.show.json
     tests/Fixtures/TicketSource/event.dates.index.json
     tests/Fixtures/TicketSource/event.show.json
     tests/Fixtures/TicketSource/event.venues.index.json
     tests/Fixtures/TicketSource/venue.dates.index.json
     tests/Fixtures/TicketSource/venue.show.json
     tests/Unit/TicketSourceAPITest.php
│
◇  Changes analyzed
│
└  ✖ OpenAI API Error: 400 - Bad Request

{
  "error": {
    "message": "This model's maximum context length is 4097 tokens. However, your messages resulted in 8332 tokens. Please reduce the length of the messages.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}
privatenumber commented 1 year ago

That's not the same error:

-    "message": "This model's maximum context length is 4097 tokens. However, you requested 5838 tokens (2909 in the messages, 2929 in the completion). Please reduce the length of the messages or completion.",
+    "message": "This model's maximum context length is 4097 tokens. However, your messages resulted in 8332 tokens. Please reduce the length of the messages.",

Previously, we were requesting more tokens than we can. Which was just fixed.

In your case, your diff is too large. Would you mind filing a new issue for this?

rocktimsaikia commented 1 year ago

@privatenumber I've been a bit busy lately. Will try to look into this soon.

sunscreem commented 1 year ago

@privatenumber Done - thank you.