Iteration through the Screener object fails

[mnalevanko]

To start with, this is the filter I apply when screening for stocks: filters = ['sh_curvol_o300', 'ta_highlow52w_b0to10h', 'ind_stocksonly']. When I print the length of the Screener object, the result is 515 (as I write this). However, when I try to iterate through the object and print the ticker symbols only, the iteration stops before it's completed. As a result, I can see only first tickers. Any idea why is this happening? Or even better: is there a working solution?

Thank you very much.

[d3an]

To start with, this is the filter I apply when screening for stocks: filters = ['sh_curvol_o300', 'ta_highlow52w_b0to10h', 'ind_stocksonly']. When I print the length of the Screener object, the result is 515 (as I write this). However, when I try to iterate through the object and print the ticker symbols only, the iteration stops before it's completed. As a result, I can see only first tickers. Any idea why is this happening? Or even better: is there a working solution?

Thank you very much.

Can you post a code sample to reproduce the problem?

[mnalevanko]

Hopefully, the following code illustrates what seems to be the issue. Although I am able to retrieve the Screener object, I am not able to iterate through it to have tickers saved in my own list. Any idea what am I missing?

from finviz.screener import Screener
filters = ['sh_curvol_o300', 'ta_highlow52w_b0to10h', 'ind_stocksonly']

_stock_list = Screener(filters=filters)
print(f'Number of stocks found: {len(_stock_list)}')
tickers = []

for stock in _stock_list:
    tickers.append(stock['Ticker'])

print(f'Number of tickers available: {len(tickers)}')

[d3an]

Here's what I've got:

from finviz.screener import Screener
filters = ['sh_curvol_o300', 'ta_highlow52w_b0to10h', 'ind_stocksonly']
_stock_list = Screener(filters=filters)
print(_stock_list)

No. | Ticker | Company                                  | Sector                 | Industry                            | Country        | Market Cap | P/E      | Price   | Change | Volume    
--- | ------ | ---------------------------------------- | ---------------------- | ----------------------------------- | -------------- | ---------- | -------- | ------- | ------ | ----------
21  | ALB    | Albemarle Corporation                    | Basic Materials        | Specialty Chemicals                 | USA            | 10.36B     | 23.42    | 96.48   | 1.60%  | 637,660   
22  | ALC    | Alcon Inc.                               | Healthcare             | Medical Instruments & Supplies      | Switzerland    | 30.49B     | -        | 62.34   | 1.91%  | 1,161,419 
23  | ALV    | Autoliv, Inc.                            | Consumer Cyclical      | Auto Parts                          | Sweden         | 7.34B      | 52.66    | 84.94   | 0.88%  | 730,671   
24  | ALXN   | Alexion Pharmaceuticals, Inc.            | Healthcare             | Biotechnology                       | USA            | 26.20B     | 32.58    | 121.61  | 1.62%  | 994,100   
25  | AMAG   | AMAG Pharmaceuticals, Inc.               | Healthcare             | Biotechnology                       | USA            | 472.03M    | -        | 13.67   | 0.07%  | 713,047   
26  | AMCR   | Amcor plc                                | Consumer Cyclical      | Packaging & Containers              | Switzerland    | 17.16B     | 28.27    | 10.97   | -0.63% | 3,024,979 
27  | AME    | AMETEK, Inc.                             | Industrials            | Specialty Industrial Machinery      | USA            | 24.56B     | 27.93    | 107.76  | 0.70%  | 880,223   
28  | AMH    | American Homes 4 Rent                    | Real Estate            | REIT - Residential                  | USA            | 9.11B      | 109.34   | 29.96   | -0.83% | 1,299,784 
29  | AMP    | Ameriprise Financial, Inc.               | Financial              | Asset Management                    | USA            | 20.62B     | 8.91     | 170.39  | 0.04%  | 492,842   
30  | AMZN   | Amazon.com, Inc.                         | Consumer Cyclical      | Internet Retail                     | USA            | 1637.38B   | 123.17   | 3204.40 | 0.88%  | 3,427,388 
31  | ANSS   | ANSYS, Inc.                              | Technology             | Software - Application              | USA            | 28.49B     | 71.97    | 330.22  | 1.05%  | 349,885   
32  | AOS    | A. O. Smith Corporation                  | Industrials            | Specialty Industrial Machinery      | USA            | 8.94B      | 30.20    | 54.93   | 0.04%  | 773,937   
33  | APAM   | Artisan Partners Asset Management Inc.   | Financial              | Asset Management                    | USA            | 3.38B      | 15.87    | 43.25   | -0.35% | 404,358   
34  | APD    | Air Products and Chemicals, Inc.         | Basic Materials        | Chemicals                           | USA            | 64.47B     | 34.08    | 294.00  | 0.60%  | 477,477   
35  | APG    | APi Group Corporation                    | Industrials            | Engineering & Construction          | USA            | 2.59B      | -        | 15.32   | 0.52%  | 551,259   
36  | APPN   | Appian Corporation                       | Technology             | Software - Infrastructure           | USA            | 5.35B      | -        | 78.28   | -1.17% | 466,328   
37  | APTV   | Aptiv PLC                                | Consumer Cyclical      | Auto Parts                          | Ireland        | 26.93B     | 15.27    | 100.29  | 0.14%  | 1,886,188 
38  | AQN    | Algonquin Power & Utilities Corp.        | Utilities              | Utilities - Renewable               | Canada         | 9.31B      | 16.39    | 15.72   | 0.58%  | 410,354   
39  | AQUA   | Evoqua Water Technologies Corp.          | Industrials            | Pollution & Treatment Controls      | USA            | 2.74B      | 33.74    | 23.65   | 0.42%  | 352,531   
40  | ARE    | Alexandria Real Estate Equities, Inc.    | Real Estate            | REIT - Office                       | USA            | 20.23B     | 49.11    | 160.54  | 0.24%  | 385,475   
41  | ARES   | Ares Management Corporation              | Financial              | Asset Management                    | USA            | 11.44B     | 99.18    | 44.83   | 1.01%  | 306,696   
42  | ARNA   | Arena Pharmaceuticals, Inc.              | Healthcare             | Biotechnology                       | USA            | 4.73B      | -        | 81.34   | 1.23%  | 336,929   
43  | ARNC   | Arconic Corporation                      | Industrials            | Metal Fabrication                   | USA            | 2.60B      | 17.72    | 23.89   | 0.59%  | 577,068   
44  | ARRY   | Array Technologies, Inc.                 | Technology             | Solar                               | USA            | 4.99B      | -        | 42.00   | -4.42% | 3,296,685 
45  | ARW    | Arrow Electronics, Inc.                  | Technology             | Electronics & Computer Distribution | USA            | 6.48B      | 17.63    | 83.49   | -0.67% | 456,394   
46  | ASML   | ASML Holding N.V.                        | Technology             | Semiconductor Equipment & Materials | Netherlands    | 161.00B    | 40.68    | 378.56  | 0.81%  | 467,477   
47  | ASO    | Academy Sports and Outdoors, Inc.        | Consumer Cyclical      | Specialty Retail                    | USA            | 1.26B      | -        | 14.58   | -1.22% | 496,672   
48  | ATHM   | Autohome Inc.                            | Communication Services | Internet Content & Information      | China          | 11.25B     | 24.62    | 96.66   | -3.08% | 364,362   
49  | ATSG   | Air Transport Services Group, Inc.       | Industrials            | Integrated Freight & Logistics      | USA            | 1.79B      | 29.27    | 29.88   | 1.98%  | 719,648   
50  | ATVI   | Activision Blizzard, Inc.                | Communication Services | Electronic Gaming & Multimedia      | USA            | 62.34B     | 34.59    | 81.04   | 0.33%  | 4,299,932 
51  | AVGO   | Broadcom Inc.                            | Technology             | Semiconductors                      | USA            | 151.28B    | 69.86    | 372.72  | -0.17% | 885,677   
52  | AVID   | Avid Technology, Inc.                    | Communication Services | Electronic Gaming & Multimedia      | USA            | 412.34M    | 30.16    | 9.35    | 5.06%  | 1,355,458 
53  | AVTR   | Avantor, Inc.                            | Basic Materials        | Specialty Chemicals                 | USA            | 14.10B     | 106.41   | 24.90   | -2.16% | 11,183,890
54  | AVXL   | Anavex Life Sciences Corp.               | Healthcare             | Biotechnology                       | USA            | 338.18M    | -        | 5.72    | 8.94%  | 2,092,509 
55  | AVY    | Avery Dennison Corporation               | Industrials            | Business Equipment & Supplies       | USA            | 11.73B     | 22.97    | 141.74  | 0.62%  | 680,662   
56  | AVYA   | Avaya Holdings Corp.                     | Technology             | Software - Application              | USA            | 1.53B      | -        | 17.99   | -3.85% | 1,013,464 
57  | AWK    | American Water Works Company, Inc.       | Utilities              | Utilities - Regulated Water         | USA            | 27.95B     | 43.71    | 154.08  | 0.31%  | 422,702   
58  | AXNX   | Axonics Modulation Technologies, Inc.    | Healthcare             | Medical Devices                     | USA            | 1.97B      | -        | 49.94   | 2.93%  | 367,271   
59  | AY     | Atlantica Sustainable Infrastructure plc | Utilities              | Utilities - Renewable               | United Kingdom | 3.06B      | 181.92   | 30.38   | 1.17%  | 376,463   
60  | BABA   | Alibaba Group Holding Limited            | Consumer Cyclical      | Internet Retail                     | China          | 817.11B    | 31.76    | 309.92  | 1.19%  | 6,220,632 
81  | BRO    | Brown & Brown, Inc.                      | Financial              | Insurance Brokers                   | USA            | 13.18B     | 30.12    | 46.72   | 0.17%  | 791,337   
82  | BSIG   | BrightSphere Investment Group Inc.       | Financial              | Asset Management                    | USA            | 1.22B      | 6.76     | 15.19   | 0.33%  | 318,998   
83  | BYD    | Boyd Gaming Corporation                  | Consumer Cyclical      | Resorts & Casinos                   | USA            | 3.76B      | -        | 34.70   | 0.61%  | 1,503,009 
84  | CAG    | Conagra Brands, Inc.                     | Consumer Defensive     | Packaged Foods                      | USA            | 18.50B     | 18.67    | 38.00   | 0.93%  | 2,127,897 
85  | CALX   | Calix, Inc.                              | Technology             | Software - Application              | USA            | 1.62B      | -        | 26.05   | 1.36%  | 935,098   
86  | CAT    | Caterpillar Inc.                         | Industrials            | Farm & Heavy Construction Machinery | USA            | 89.29B     | 22.71    | 168.59  | -0.63% | 2,848,034 
87  | CC     | The Chemours Company                     | Basic Materials        | Specialty Chemicals                 | USA            | 3.58B      | -        | 21.88   | 0.88%  | 556,543   
88  | CCCC   | C4 Therapeutics, Inc.                    | Healthcare             | Biotechnology                       | USA            | 1.09B      | -        | 26.09   | -2.83% | 428,537   
89  | CCIV   | Churchill Capital Corp IV                | Financial              | Shell Companies                     | USA            | 1.99B      | -        | 9.63    | -0.21% | 2,067,550 
90  | CCK    | Crown Holdings, Inc.                     | Consumer Cyclical      | Packaging & Containers              | USA            | 12.06B     | 25.16    | 90.38   | -1.16% | 1,043,833 
91  | CCS    | Century Communities, Inc.                | Consumer Cyclical      | Residential Construction            | USA            | 1.49B      | 9.94     | 44.01   | 4.54%  | 332,115   
92  | CDAY   | Ceridian HCM Holding Inc.                | Technology             | Software - Application              | USA            | 13.55B     | 183.32   | 92.21   | 2.96%  | 766,497   
93  | CDNA   | CareDx, Inc                              | Healthcare             | Diagnostics & Research              | USA            | 2.49B      | -        | 50.18   | -1.12% | 328,344   
94  | CDNS   | Cadence Design Systems, Inc.             | Technology             | Software - Application              | USA            | 31.31B     | 30.85    | 112.05  | 2.50%  | 1,065,967 
95  | CE     | Celanese Corporation                     | Basic Materials        | Chemicals                           | USA            | 14.15B     | 22.53    | 120.18  | 1.79%  | 1,083,653 
96  | CERN   | Cerner Corporation                       | Healthcare             | Health Information Services         | USA            | 22.25B     | 44.46    | 73.72   | 1.56%  | 847,895   
97  | CGNX   | Cognex Corporation                       | Technology             | Scientific & Technical Instruments  | USA            | 11.81B     | 86.76    | 69.93   | 0.45%  | 430,757   
98  | CHD    | Church & Dwight Co., Inc.                | Consumer Defensive     | Household & Personal Products       | USA            | 23.20B     | 32.05    | 91.91   | 1.01%  | 957,878   
99  | CHGG   | Chegg, Inc.                              | Consumer Defensive     | Education & Training Services       | USA            | 10.74B     | 17352.00 | 86.76   | 0.24%  | 1,897,805 
100 | CHKP   | Check Point Software Technologies Ltd.   | Technology             | Software - Application              | Israel         | 17.03B     | 21.16    | 120.47  | -0.44% | 953,816   
381 | RCII   | Rent-A-Center, Inc.                      | Industrials            | Rental & Leasing Services           | USA            | 1.77B      | 11.50    | 32.76   | 1.42%  | 413,848   
382 | RCKT   | Rocket Pharmaceuticals, Inc.             | Healthcare             | Biotechnology                       | USA            | 1.56B      | -        | 28.10   | -2.45% | 304,440   
383 | RCM    | R1 RCM Inc.                              | Healthcare             | Health Information Services         | USA            | 2.04B      | 171.27   | 17.47   | 0.46%  | 600,684   
384 | REPL   | Replimune Group, Inc.                    | Healthcare             | Biotechnology                       | USA            | 1.82B      | -        | 45.20   | 4.17%  | 1,400,835 
385 | RESI   | Front Yard Residential Corporation       | Real Estate            | REIT - Residential                  | USA            | 778.05M    | -        | 13.38   | 0.45%  | 918,355   
386 | REXR   | Rexford Industrial Realty, Inc.          | Real Estate            | REIT - Industrial                   | USA            | 6.02B      | 105.72   | 48.21   | 0.33%  | 493,836   
387 | RH     | RH                                       | Consumer Cyclical      | Specialty Retail                    | USA            | 7.78B      | 47.12    | 394.08  | 4.05%  | 639,683   
388 | ROK    | Rockwell Automation, Inc.                | Industrials            | Specialty Industrial Machinery      | USA            | 28.01B     | 36.87    | 243.10  | 1.58%  | 562,222   
389 | ROKU   | Roku, Inc.                               | Communication Services | Entertainment                       | USA            | 28.45B     | -        | 223.96  | 0.72%  | 2,136,579 
390 | ROL    | Rollins, Inc.                            | Consumer Cyclical      | Personal Services                   | USA            | 19.63B     | 92.76    | 60.20   | 1.26%  | 758,540   
391 | RPD    | Rapid7, Inc.                             | Technology             | Software - Application              | USA            | 3.34B      | -        | 65.71   | 2.40%  | 381,492   
392 | RPM    | RPM International Inc.                   | Basic Materials        | Specialty Chemicals                 | USA            | 11.61B     | 30.89    | 89.70   | 0.63%  | 471,450   
393 | RRC    | Range Resources Corporation              | Energy                 | Oil & Gas E&P                       | USA            | 2.43B      | -        | 9.09    | -3.19% | 3,667,773 
394 | RS     | Reliance Steel & Aluminum Co.            | Basic Materials        | Steel                               | USA            | 7.09B      | 15.88    | 111.24  | 0.45%  | 374,862   
395 | RSG    | Republic Services, Inc.                  | Industrials            | Waste Management                    | USA            | 29.53B     | 28.09    | 92.87   | 1.20%  | 696,889   
396 | RTRX   | Retrophin, Inc.                          | Healthcare             | Biotechnology                       | USA            | 1.04B      | -        | 20.95   | -0.62% | 405,474   
397 | RXN    | Rexnord Corporation                      | Industrials            | Specialty Industrial Machinery      | USA            | 3.96B      | 25.15    | 32.64   | 0.83%  | 404,370   
398 | RYAAY  | Ryanair Holdings plc                     | Industrials            | Airlines                            | Ireland        | 16.80B     | 85.39    | 89.75   | 1.70%  | 508,473   
399 | SAIL   | SailPoint Technologies Holdings, Inc.    | Technology             | Software - Infrastructure           | USA            | 4.24B      | 1110.24  | 45.52   | 0.75%  | 820,329   
400 | SBAC   | SBA Communications Corporation           | Real Estate            | REIT - Specialty                    | USA            | 33.85B     | -        | 302.11  | 2.75%  | 631,832   

I think the issue might have something to do with the async requests. There may not be enough of a delay, and as a result, Finviz may block some of them.

[d3an]

I've taken a look at Mario's code, and as long as I kept the async requests, the error kept occurring. At a lower level. the HTML returns b'Too many requests.' instead of the table we want. For the example detailed above, there are 25 requests being made (all at the same time).

The best solution for this would be to remove async requests, and institute sequential requests with a set delay to avoid setting off any alarms. It might take a bit longer to get your data, but it should maintain the integrity. I've just test 5 second delays, which work, but I'm going to see if I can reduce it. I'm also going to add an error message to avoid the silent failure.