robvankeilegom
commented
2 years ago @m4t7 Thanks for the issue. There was indeed a typo & i fixed it for you. If you run git pull to get the latest version you'll be alright. Let me know how it goes.
Closed m4t7 closed 2 years ago
robvankeilegom
commented
2 years ago @m4t7 Thanks for the issue. There was indeed a typo & i fixed it for you. If you run git pull to get the latest version you'll be alright. Let me know how it goes.
m4t7
commented
2 years ago @robvankeilegom thanks for your quick reply. The inital issue is solved, now I run into new issues. :/
[user@manjaro firefly-III-paypal-importer]$ php artisan sync
Start pulling data from PayPal
In Connection.php line 712:
could not find driver (SQL: select * from "payers" where ("pp_id" = 46VJKHTKQ5QPA) limit 1)
In Connector.php line 70:
could not find driver Here is my .env File, I assume the DB connection is not working as expected
robvankeilegom
commented
2 years ago @m4t7 This means your local environment isn't configured to support sqlite. If you're on a Debian based system like Ubuntu you'll have to install php-sqlite with sudo apt install php-sqlite. After that you'll have to restart your web server.
I'm planning to put the application in a docker container like the official firefly-importer once i have some free time. So you'll be able to start it with a single command.
m4t7
commented
2 years ago @robvankeilegom ok thanks for that explanation.
I'm using Arch Linux (Manjaro) as my daily driver and normal work environment. I installed php-sqlite from official manjaro repository. Unfortunatly I get the same error. I don't know what I forget. Do I need a websever on top? I thought with composer everything was installed.
I have specified the full path for the database in the environment file and should have every dependency.
robvankeilegom
commented
2 years ago @m4t7 Is was wrong about the web server. You're running php from the CLI, so you won't have to restart a web server. I added the necessary files to run the app in Docker.
The README.md file was updated as well. Make sure the DB_DATABASE field in your .env file has exactly the same value as in .env.example before building the container.
m4t7
commented
2 years ago @robvankeilegom
Thanks for your effort, but it still fails.
I adjusted your Dockerfile and docker-compse.yml a little bit to get my own CA_CERT installed while the container is created and map the database path within the docker-compose.
volumes:
- ./database:/var/www/htmlThis code maps the database path from the actual folder e.g. "firefly-III-paypal-importer/database". The .env File still contains /var/www/html/database.sqlite
$ cat docker-compose.yml
version: '3'
services:
firefly-iii-paypal-importer:
build:
context: .
dockerfile: Dockerfile
container_name: firefly-iii-paypal-importer
restart: unless-stopped
tty: true
working_dir: /var/www/html
volumes:
- ./database:/var/www/htmlI added at the end of the Dockerfile two RUN actions
RUN cp /var/www/html/database/CA_CERT.crt /usr/local/share/ca-certificates/
RUN /usr/sbin/update-ca-certificates$ docker exec firefly-iii-paypal-importer php artisan sync
Could not open input file: artisan
$ docker exec -it firefly-iii-paypal-importer bash
root@013c204d1248:/var/www/html# php artisan sync
Could not open input file: artisan
root@013c204d1248:/var/www/html# ls
CA_CERT.crt database.sqlite factories migrations seeders
Oh man my fault, I added the false volume.
volumes:
- ./database:/var/www/html/databaseEnvironment File has the value from example.
$ cat .env | grep DB_DA
DB_DATABASE=/var/www/html/database/database.sqlite$ docker exec firefly-iii-paypal-importer php artisan sync
Start pulling data from PayPal
In Connection.php line 712:
SQLSTATE[HY000]: General error: 1 no such table: payers (SQL: select * from
"payers" where ("pp_id" = 46VJKHTKQ5QPA) limit 1)
In Connection.php line 368:
SQLSTATE[HY000]: General error: 1 no such table: payers $ docker exec -it firefly-iii-paypal-importer bash
root@4481b8df472c:/var/www/html# ls
Dockerfile app bootstrap composer.lock database phpunit.xml resources storage vendor
README.md artisan composer.json config docker-compose.yml public routes tests
root@4481b8df472c:/var/www/html# cd database/
root@4481b8df472c:/var/www/html/database# ls
CA-CERT.crt database.sqlite factories migrations seedersNo payers file is generated. Do you see why?
robvankeilegom
commented
2 years ago @m4t7 You're getting there. You'll have to run php artisan migrate --force to generate the table you need.
m4t7
commented
2 years ago @robvankeilegom still failing , this time a new issue in Sync.php
$ docker exec -it firefly-iii-paypal-importer bash
root@4481b8df472c:/var/www/html# php artisan migrate --force
Migration table created successfully.
Migrating: 2022_02_24_205513_create_payers_table
Migrated: 2022_02_24_205513_create_payers_table (15.48ms)
Migrating: 2022_02_24_205754_create_transactions_table
Migrated: 2022_02_24_205754_create_transactions_table (18.22ms)
root@4481b8df472c:/var/www/html# php artisan sync
Start pulling data from PayPal
In Sync.php line 65:
Undefined property: stdClass::$item_details Good day @robvankeilegom hope all is well. Regarding the issue that I'm facing, I do not understand why this happens.
Source Code in app/Sync.php:
65 $cartItems = ($record->cart_info->item_details);
66
67 $transaction = Transaction::updateOrCreate([
68 'pp_id' => $record->transaction_info->transaction_id,
69 ], [
70 'reference_id' => $reference,
71 'event_code' => $record->transaction_info->transaction_event_code,
72 'initiation_date' => $record->transaction_info->transaction_initiation_date,
73 'currency' => $record->transaction_info->transaction_amount->currency_code,
74 'value' => $record->transaction_info->transaction_amount->value,
75 'description' => implode(', ', array_column($cartItems, 'item_name')),
76 ]);In the developer documentation item_details array is specified and should contain item_name that is used to populate the description. Is this correct?
https://developer.paypal.com/api/transaction-search/v1/#definition-item_detail
Here is my sqlite database
Do you see any issue?
robvankeilegom
commented
2 years ago I'm sorry for taking so long to respond. I looked into the documentation last week and concluded the same as you. I'm not sure why the call isn't returning an item_detail field. It's only used for the transaction description so we could prolly omit it.
If you're willing to debug this further you could make an api call with a tool like Postman and check out the transaction. The call requires a start_date parameter where you can pass the oldest that thats visible in your sqlite database. This application retrieves transaction from newest to oldest, so the problem should be the transaction right after the last transaction in your database.
I'll change the code to omit the description if the item_detail isn't available later this week.
m4t7
commented
2 years ago No worries.
I looked into my paypal account and I see that the app uses the payer name (e.g. alternative full name) as the transaction description if no transaction_note is found. (I assume the transaction_note is populated into the item_name)
In my example below the transaction contains an item_name":"2000640935" but that info is not useful at all and oblivious not imported into the transaction database table. (also not used in the app)
It would be great if you could check if the database contains a description value, and if not then use the alternate_full_name field and maybe extend it with the invoice_id , that mostly contain more useful information.
"payer_name":{"alternate_full_name":"Food GmbH"},
and
"invoice_id":"W21FEIKHY24RWPF5"
Food GmbH - Invoice ID: W21FEIKHY24RWPF5
robvankeilegom
commented
2 years ago @m4t7 Thanks for all the info, you've been really helpful. I pushed a change that prepends the invoice_id of the transaction to the description in Firefly. I've also added a check to see if item_details is available. Let me know how it goes!
m4t7
commented
2 years ago Thanks for all your work. The item_details issue is solved.
I looked into my database and it seems to look good but the payer_name is not extended all the time for records. I will look tomorrow with rest again over some transactions and will let you know.
Maybe you find something regarding the new issue I receive:
Start pulling data from PayPal
In Sync.php line 53:
Undefined property: stdClass::$email_address I pushed a change that makes the email_adress optionally. I checked my records and all 400 transactions I synced from the PayPal api have an email address and name.
The names are populated with payer_info.payer_name.alternate_full_name or payer_info.payer_name.given_name whichever isn't empty. If you can get a hold of a record that isn't populated correctly lmk what field i should be using.
m4t7
commented
2 years ago Thank you very very much for all your fixes so far.
The database contains now every transaction. (over 700)
I found the pp_id without any email address, it was paypal itself^^ (I think email is today mandatory , maybe pp accounts that where opened before a specifc date may contain this blank email)
Here is an example of a transaction that do not contain the payer_name as alternative if no item_name is assigend.
The transaction with payer_id 4 had initial no description, then your fix added the invoice_id , but it failed to add the payer_name. It would be great if you can check this again.
The transaction with pp_id and reference_id contains no alternative full name
The transaction with only the reference_id contains the alternative_full_name but it is still not populated into the DB as shown above.
Since I have all transactions, the sync now completes the pulling phase and moves into the pushing phase.
root@8562e6701a2c:/var/www/html# php artisan sync
Start pulling data from PayPal
Pushing data to Firefly
In Firefly.php line 69:
Call to undefined method GuzzleHttp\Exception\ConnectException::getResponse() I'm facing a new issue, is this really undefined or is this any thing other? I found nothing in the storage log.
robvankeilegom
commented
2 years ago @m4t7 I messed up there, that should be fixed now. Make sure the FIREFLY_URL value in your .env file just the root domain without a path.
I purposely didn't populate the transaction description with the payer name. The description will be empty if no invoice_id or items names are returned from the PayPal api but that's fine. In Firefly you'll always see the payer alongside the transaction.
m4t7
commented
2 years ago @robvankeilegom yes you fixed it. I agree with you on that, the import with incoice_id or items_names is completely fine.It looks really fine in firefly :+1:
Unfortunately after 7 entries the import into firefly stops with a new issue.
I have all 5 transactions from march and my first 2 transactions from february.
root@7bc27f9fbf27:/var/www/html# php artisan sync
Start pulling data from PayPal
Pushing data to Firefly
In RequestException.php line 113:
Client error: `POST https://cash.mt.rpn/api/v1/accounts` resulted in a `422 Unprocessable Entity` response:
{"message":"The given data was invalid.","errors":{"name":["This account name is already in use."]}} @m4t7 I created #4 for this.
Good day,
this project seems interessting. I followed your install instructions.
Unfortunatly I get the following issue after running artisan sync
Same error is shown in the log.
storage/logs/lumen-2022-03-05.log
``` #0 /home/user/github/firefly-III-paypal-importer/app/Console/Commands/Sync.php(31): Illuminate\\Console\\Command->__call() #1 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/container/BoundMethod.php(36): App\\Console\\Commands\\Sync->handle() #2 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/container/Util.php(40): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}() #3 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure() #4 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod() #5 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/container/Container.php(653): Illuminate\\Container\\BoundMethod::call() #6 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/console/Command.php(136): Illuminate\\Container\\Container->call() #7 /home/user/github/firefly-III-paypal-importer/vendor/symfony/console/Command/Command.php(298): Illuminate\\Console\\Command->execute() #8 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/console/Command.php(121): Symfony\\Component\\Console\\Command\\Command->run() #9 /home/user/github/firefly-III-paypal-importer/vendor/symfony/console/Application.php(1015): Illuminate\\Console\\Command->run() #10 /home/user/github/firefly-III-paypal-importer/vendor/symfony/console/Application.php(299): Symfony\\Component\\Console\\Application->doRunCommand() #11 /home/user/github/firefly-III-paypal-importer/vendor/symfony/console/Application.php(171): Symfony\\Component\\Console\\Application->doRun() #12 /home/user/github/firefly-III-paypal-importer/vendor/illuminate/console/Application.php(94): Symfony\\Component\\Console\\Application->run() #13 /home/user/github/firefly-III-paypal-importer/vendor/laravel/lumen-framework/src/Console/Kernel.php(116): Illuminate\\Console\\Application->run() #14 /home/user/github/firefly-III-paypal-importer/artisan(35): Laravel\\Lumen\\Console\\Kernel->handle() #15 {main} "} [2022-03-05 12:28:16] local.ERROR: Method App\Console\Commands\Sync::syncPayPal does not exist. {"exception":"[object] (BadMethodCallException(code: 0): Method App\\Console\\Commands\\Sync::syncPayPal does not exist. at /home/user/github/firefly-III-paypal-importer/vendor/illuminate/macroable/Traits/Macroable.php:113) [stacktrace] ```Macroable.php function that contains line 113:
I assume Sync::syncPayPal is unspecified or not on the expected location.
If I can debug or otherwise be helpful let me know.