Open gitman2015 opened 2 years ago
Finally, can you please add the Eni provider (https://www.editions-eni.fr/)?
I create a new issue for it: #5
I got a psr-18 error when I test your example:
The library don't come with any PSR-18 implementation.
The goal is not to force anyone to use the implementation I choose, but to let anyone you their preferred PSR-18 library.
You can pick up any library from this list: https://packagist.org/providers/psr/http-client-implementation
For example guzzlehttp/guzzle
is a good candidate as it provide PSR-18, PSR-7 and PSR-17 implementation (which are needed by this library) and is a well known and maintained package.
Also, can you please show a complete working example. I don't know what to put in
$providerConfiguration = ...;
either in Configurable provider or Multiple providers.
Here a working example of a Provider Configuration: https://github.com/MacFJA/livres/blob/master/src/Repository/ProviderConfigurationRepository.php
But if the provider(s) you want to use have no particular configuration, you can simply write this:
$providerConfiguration = new class implements \MacFJA\BookRetriever\ProviderConfigurationInterface {
public function getParameters(\MacFJA\BookRetriever\ProviderInterface $provider): array
{
return [];
}
public function isActive(\MacFJA\BookRetriever\ProviderInterface $provider): bool
{
return true;
}
};
The $providerConfiguration
(that need to implement the interface ProviderConfigurationInterface
) allow you to dynamically add options to a provider (like API token)
I hope that answer your question, If not don't hesitate to ask more details
Thanks for your answer
There is already the "guzzlehttp/guzzle" package when launches the composer require statement:
PS C:\XAMPP\htdocs\blog\Book-Retriever> **composer require macfja/book-retriever**
Using version ^1.1 for macfja/book-retriever
./composer.json has been updated
Running composer update macfja/book-retriever
Loading composer repositories with package information
Updating dependencies
Lock file operations: 17 installs, 0 updates, 0 removals
- Locking cedcommerce/ebay-sdk-php (18.0.1)
- Locking fale/isbn (3.1.0)
- **Locking guzzlehttp/guzzle** (6.5.5)
- Locking guzzlehttp/promises (1.5.1)
- Locking guzzlehttp/psr7 (1.8.3)
- Locking macfja/book-retriever (1.1.1)
- Locking masterminds/html5 (2.7.5)
- Locking njt/good-reads (1.0.0)
- Locking php-http/discovery (1.14.1)
- Locking psr/http-client (1.0.1)
- Locking psr/http-factory (1.0.1)
- Locking psr/http-message (1.0.1)
- Locking ralouphie/getallheaders (3.0.3)
- Locking scriptotek/google-books (v0.2.5)
- Locking symfony/polyfill-intl-idn (v1.25.0)
- Locking symfony/polyfill-intl-normalizer (v1.25.0)
- Locking symfony/polyfill-php72 (v1.25.0)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 17 installs, 0 updates, 0 removals
- Installing guzzlehttp/promises (1.5.1): Extracting archive
- Installing ralouphie/getallheaders (3.0.3): Extracting archive
- Installing psr/http-message (1.0.1): Extracting archive
- Installing guzzlehttp/psr7 (1.8.3): Extracting archive
- Installing symfony/polyfill-php72 (v1.25.0): Extracting archive
- Installing symfony/polyfill-intl-normalizer (v1.25.0): Extracting archive
- Installing symfony/polyfill-intl-idn (v1.25.0): Extracting archive
- **Installing guzzlehttp/guzzle (6.5.5)**: Extracting archive
- Installing scriptotek/google-books (v0.2.5): Extracting archive
- Installing psr/http-factory (1.0.1): Extracting archive
- Installing psr/http-client (1.0.1): Extracting archive
- Installing php-http/discovery (1.14.1): Extracting archive
- Installing njt/good-reads (1.0.0): Extracting archive
- Installing masterminds/html5 (2.7.5): Extracting archive
- Installing fale/isbn (3.1.0): Extracting archive
- Installing cedcommerce/ebay-sdk-php (18.0.1): Extracting archive
- Installing macfja/book-retriever (1.1.1): Extracting archive
3 package suggestions were added by new dependencies, use `composer suggest` to see details.
Generating autoload files
5 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
I tested a lot of package without success (guzzlehttp/guzzle + guzzle/guzzle6-adapter ...), now I have no more error with this composer.json:
{
"require": {
"macfja/book-retriever": "^1.1",
"symfony/http-client": "^5.4",
"php-http/httplug": "^2.3",
"nyholm/psr7": "^1.5",
"symfony/var-dumper": "^5.4"
}
}
Can you test in a new project your library, just the simple example please (provider AntoineOnline) ?
I don't get any results with isbn: 9782253006329
$htmlGetter = new \MacFJA\BookRetriever\Helper\HtmlGetter();
$isbnTool = new \Isbn\Isbn();
$antoineOnline = new \MacFJA\BookRetriever\Provider\AntoineOnline($htmlGetter, $isbnTool);
$books = $antoineOnline->searchIsbn('9782253006329');
In your AntoinOnline.php file,
$location = $response->getHeader('location');
returns an empty array
There's no key location in headers: $headers=$response->getHeaders();
However I noticed that there is a location key in response_headers through the library:
^ Nyholm\Psr7\Response {[#34 ▼]()
-reasonPhrase: "OK"
-statusCode: 200
-headers: array:8 [[▶]()]
-headerNames: array:8 [[▶]()]
-protocol: "1.1"
**-stream: Nyholm\Psr7\Stream** {[#36 ▼]()
-stream: stream resource [@17 ▼]()
timed_out: false
blocked: true
eof: false
wrapper_data: Symfony\Component\HttpClient\Response\StreamWrapper {[#37 ▼]()
+context: null
-client: Symfony\Component\HttpClient\Response\CurlResponse {[#31 ▼](http://localhost:3000/#sf-dump-1145398349-ref231)
**response_headers**: array:24 [[▼]()
0 => "HTTP/1.1 302 Found"
1 => "Cache-Control: private"
2 => "Content-Type: text/html; charset=utf-8"
3 => "ETag: """
**4 => "Location:** /Book_Vingt_Mille_Lieues_Sous_Les_Mers_Jules_Verne_9782253006329.aspx?productCode=0009782253006329"
5 => "Server: Microsoft-IIS/8.5"
6 => "X-AspNet-Version: 2.0.50727"
7 => "X-Powered-By: UrlRewriter.NET 1.7.0"
8 => "Set-Cookie: ASP.NET_SessionId=kcnz4m451wqrd1550od24v55; path=/; HttpOnly"
9 => "Set-Cookie: newAntoineOnlineLanguage=GrysQiWsf1yG1blDuXn4MOt8I62BA2Oxqe46sJVp3MfMd_sJvDQXrWzj6JaWyX2iFxJtHAbxBBfET1pibxvw12HlV1I1; expires=Mon, 14-Mar-2022 09:0[ ▶]()"
10 => "Set-Cookie: newAntoineOnlineLanguage=AllnlMvScWi6-urFWHKiP6U1ZGwxTVsnTiX5LM5moUNq7aVj1vqpLerc5uQsconjUCYrHKoRS4KK6MnOAiTj8j5RfaU1; expires=Mon, 14-Mar-2022 09:0[ ▶]()"
11 => "X-Powered-By: ASP.NET"
12 => "Date: Sun, 13 Mar 2022 09:03:18 GMT"
13 => "Content-Length: 221"
14 => "HTTP/1.1 200 OK"
15 => "Cache-Control: private"
16 => "Content-Type: text/html; charset=utf-8"
I don't know how to retrieve the 'location' from the stream.
the second problem is that if we have a 200 response, you return an empty array (in AntoineOnline.php)
if ($response->getStatusCode() < 300 || $response->getStatusCode() > 399) {
return [];
}
Finally, if I don't take location into account for testing purposes, I get a result in SearchBuilder:
^ "searchResultBuilder:"
^ array:1 [[▼]()
0 => MacFJA\BookRetriever\SearchResult\SearchResult {[#69 ▼]()
-isbn: "9782253006329"
-title: "Vingt mille lieues sous les mers"
-authors: array:1 [[▶]()]
-pages: null
-series: null
-illustrators: []
-translators: []
-genres: []
-publicationDate: null
-format: null
-dimension: null
-keywords: []
-cover: "https://antoine-images.com/imagerepository.ashx?ean=9782253006329&SIZE=l&WebSite=AO"
-additional: array:1 [[▼]()
"antoineonline_link" => array:1 [[▼]()
0 => "https://www.antoineonline.com"
]
]
}
]
Here is the modified AmazonOnline.php file to override location and comment out the getStatusCode:
public function searchIsbn(string $isbn): array
{
$isbn = $this->isbnTool->hyphens->removeHyphens($isbn);
// dump($isbn);
$client = $this->getHttpClient();
// dump($client);
$requestBody = sprintf(static::WEBPAGE_SEARCH_PATTERN, urlencode($isbn));
dump("$requestBody:", $requestBody);
$request = $this->createHttpRequest('POST', $requestBody)
->withAddedHeader('content-length', (string) strlen($requestBody));
dump($request);
$response = $client->sendRequest($request);
// location est dans response
dump('response:',$response);
// dump('getBody',$response->getBody()->getContents());
// echo ($response->getBody()->getContents());
dump($response->getStatusCode());
// if ($response->getStatusCode() < 300 || $response->getStatusCode() > 399) {
// return [];
// }
$url = "https://www.antoineonline.com/Livre_Vingt_Mille_Lieues_Sous_Les_Mers_Jules_Verne_9782253006329.aspx?productCode=0009782253006329";
// new NativeResponse($url,null,'',[],null,null,null,null);
// ATTENTION: à ce stade on ne RECCUPERE RIEN de getHeader('location)
$location = $response->getHeader('location');
$headers=$response->getHeaders();
dump('headers:', $headers);
dump('location:',$location);
// new StreamWrapper();
$location = reset($location);
// $xml = $this->getHtmlGetter()->getWebpageAsDom(static::WEBPAGE_BASE_URL.$location);
$xml = $this->getHtmlGetter()->getWebpageAsDom($url);
// dump('xml:',$xml);
$allMeta = $xml->getElementsByTagName('meta');
dump('allMeta:',$allMeta);
$resultDiv = $xml->getElementById('ctl00_cph1_ProductMainPage');
// $resultDiv = $xml->getElementById('ctl00_cph1_RandomNumbers');
dump('resultDiv:',$resultDiv);
if (null === $resultDiv) {
return [];
}
$allInput = $resultDiv->getElementsByTagName('input');
dump('allInput:',$allInput);
$result = ['antoineonline_link' => static::WEBPAGE_BASE_URL.$location, 'isbn' => $isbn];
$result = $this->handleNode($result, $allMeta, [$this, 'metaNodeVisitor']);
$result = $this->handleNode($result, $allInput, [$this, 'inputNodeVisitor']);
dump('searchResultBuilder:',[SearchResultBuilder::createFromArray($result)]);
return [SearchResultBuilder::createFromArray($result)];
}
For the amazon provider, I think the constant:
protected const API_BASE_URL_PATTERN = 'http://webservices.amazon...'
is no longer valid and should be replaced with https://aws.amazon.com...
I would like to test another provider but for the moment only AntoineOnline displays a result.
I would like to have examples that work with other providers, please, because I don't know what parameters to put when they are necessary
also when i test with $provider Configuration it doesn't work:
$providerConfiguration = new class implements \MacFJA\BookRetriever\ProviderConfigurationInterface {
public function getParameters(\MacFJA\BookRetriever\ProviderInterface $provider): array
{
return [];
}
public function isActive(\MacFJA\BookRetriever\ProviderInterface $provider): bool
{
return true;
}
};
$configurator = new \MacFJA\BookRetriever\ProviderConfigurator($providerConfiguration,null,null,null);
$htmlGetter = new \MacFJA\BookRetriever\Helper\HtmlGetter();
$isbn = new \Isbn\Isbn();
$opds = new \MacFJA\BookRetriever\Helper\OPDSParser();
$sru = new \MacFJA\BookRetriever\Helper\SRUParser();
$providers = [
// new \MacFJA\BookRetriever\Provider\AbeBooks($htmlGetter),
// new \MacFJA\BookRetriever\Provider\Amazon(),
new \MacFJA\BookRetriever\Provider\AntoineOnline($htmlGetter, $isbn),
// new \MacFJA\BookRetriever\Provider\ArchiveOrg($opds),
// new \MacFJA\BookRetriever\Provider\LibraryHub($sru),
// new \MacFJA\BookRetriever\Provider\DigitEyes(),
// new \MacFJA\BookRetriever\Provider\Ebay()
];
array_walk($providers, [$configurator, 'configure']);
$pool = new \MacFJA\BookRetriever\Pool($providers, $providerConfiguration);
$books = $pool->searchIsbn('9782253006329');
I get this error:
<html><body>
<!--StartFragment--><font size="1">
( ! ) Warning: assert(): assert($providers instanceof Traversable) failed in C:\XAMPP\htdocs\blog\Book-Retriever\vendor\macfja\book-retriever\lib\Pool.php on line 109
--
1 | 0.0006 | 356584 | {main}( ) | ...\index.php:0
2 | 3.4582 | 731160 | MacFJA\BookRetriever\Pool->searchIsbn( $isbn = '9782253006329' ) | ...\index.php:58
3 | 3.4582 | 731160 | MacFJA\BookRetriever\Pool->getActiveProviders( ) | ...\Pool.php:71
4 | 3.4582 | 731160 | assert( $assertion = FALSE, $description = 'assert($providers instanceof Traversable)' ) | ...\Pool.php:109
</font><!--EndFragment-->
</body>
</html>( ! ) Warning: [assert](http://www.php.net/function.assert)(): assert($providers instanceof Traversable) failed in C:\XAMPP\htdocs\blog\Book-Retriever\vendor\macfja\book-retriever\lib\Pool.php on line 109
Call Stack
# Time Memory Function Location
1 0.0006 356584 {main}( ) ...\index.php:0
2 3.4582 731160 MacFJA\BookRetriever\Pool->searchIsbn( $isbn = '9782253006329' ) ...\index.php:58
3 3.4582 731160 MacFJA\BookRetriever\Pool->getActiveProviders( ) ...\Pool.php:71
4 3.4582 731160 assert( $assertion = FALSE, $description = 'assert($providers instanceof Traversable)' ) ...\Pool.php:109
( ! ) Fatal error: Uncaught TypeError: Argument 1 passed to IteratorIterator::__construct() must implement interface Traversable, array given in C:\XAMPP\htdocs\blog\Book-Retriever\vendor\macfja\book-retriever\lib\Pool.php on line 111
( ! ) TypeError: Argument 1 passed to IteratorIterator::__construct() must implement interface Traversable, array given in C:\XAMPP\htdocs\blog\Book-Retriever\vendor\macfja\book-retriever\lib\Pool.php on line 111
Call Stack
# Time Memory Function Location
1 0.0006 356584 {main}( ) ...\index.php:0
2 3.4582 731160 MacFJA\BookRetriever\Pool->searchIsbn( $isbn = '9782253006329' ) ...\index.php:58
3 3.4582 731160 MacFJA\BookRetriever\Pool->getActiveProviders( ) ...\Pool.php:71
There's no key location in headers:
$headers=$response->getHeaders();
For the amazon provider, I think the constant:
protected const API_BASE_URL_PATTERN = 'http://webservices.amazon...'
is no longer valid and should be replaced withhttps://aws.amazon.com...
I didn't updated the library for almost 2 years, so it's highly probable that some provider have changed their API or about data are displayed on their website.
I don't know what parameters to put when they are necessary
I will update the [Providers.md]() file to add more details about the required parameter of the providers that need them. I will also create an Exception to indicate the missing parameters for those providers
I would like to have examples that work with other providers
For some example you can look at the Tests/Unit directory.
But I will create a full working example (other that https://github.com/MacFJA/livres)
Hello, I got a psr-18 error when I test your example:
No PSR-18 clients found. Make sure to install a package providing "psr/http-client-implementation". Example: "php-http/guzzle6-adapter".
I'm testing your library outside the symfony framework, just in a pure php project
Also, can you please show a complete working example. I don't know what to put in
$providerConfiguration = ...;
either in Configurable provider or Multiple providers.Finally, can you please add the Eni provider (https://www.editions-eni.fr/)?
I can't wait to test all these features which seem very complete and powerful to me! Thanks for your help
my composer.json: