[firebase_app_check, firebase_vertexai]: Firebase App Check Not Working for Flutter Web, Vertex AI SDK (Gemini API)

garysm commented 4 months ago

Is there an existing issue for this?

[X] I have searched the existing issues.

Which plugins are affected?

App Check, Other

Which platforms are affected?

Web

Description

Attempting to enforce Firebase App Check on Vertex AI SDK in a Flutter Web app. Confirmed setup of reCAPTCHA v3 secret and site tokens. App Check works when calling Firebase Storage within the same web application.

Reproducing the issue

Initialize Firebase App Check with reCAPTCHA v3, make call to Vertex AI (gemini 1.5 flash). See example below:

import 'package:flutter_test/firebase_options.dart';
import 'package:firebase_app_check/firebase_app_check.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_vertexai/firebase_vertexai.dart';
import 'package:flutter/material.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp(
    options: DefaultFirebaseOptions.currentPlatform,
  );
  await FirebaseAppCheck.instance.activate(
    webProvider: ReCaptchaV3Provider('recaptcha-v3-site-key'),
  );
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: Scaffold(
        body: Center(
          child: ElevatedButton(
            onPressed: () async {
              final instance = FirebaseVertexAI.instance;
              final model = instance.generativeModel(model: 'gemini-1.5-flash');
              final response = await model.generateContent(
                [
                  Content.multi([
                    DataPart('application/pdf', fileData),
                    TextPart('Tell a story about this document')
                  ])
                ],
                generationConfig: GenerationConfig(
                  temperature: 0,
                ),
              );
              print(response.text);
            },
            child: const Text('Get response'),
          ),
        ),
      ),
    );
  }
}

Firebase Core version

3.2.0

Flutter Version

3.24.0-1.0.pre.226

Relevant Log Output

{
    "error": {
        "code": 401,
        "message": "Firebase App Check token is invalid.",
        "status": "UNAUTHENTICATED"
    }
}

Flutter dependencies

Expand Flutter dependencies snippet

```yaml Dart SDK 3.6.0-70.0.dev Flutter SDK 3.24.0-1.0.pre.226 my_app 1.0.0+1 dependencies: - cupertino_icons 1.0.8 - file_picker 8.0.6 [flutter flutter_web_plugins flutter_plugin_android_lifecycle plugin_platform_interface ffi path win32 cross_file web] - firebase_app_check 0.3.0+3 [firebase_app_check_platform_interface firebase_app_check_web firebase_core firebase_core_platform_interface flutter] - firebase_auth 5.1.2 [firebase_auth_platform_interface firebase_auth_web firebase_core firebase_core_platform_interface flutter meta] - firebase_core 3.2.0 [firebase_core_platform_interface firebase_core_web flutter meta] - firebase_storage 12.1.1 [firebase_core firebase_core_platform_interface firebase_storage_platform_interface firebase_storage_web flutter] - firebase_vertexai 0.2.2+2 [firebase_app_check firebase_auth firebase_core firebase_core_platform_interface flutter google_generative_ai] - flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine] - flutter_hooks 0.20.5 [flutter] - freezed_annotation 2.4.4 [collection json_annotation meta] - hooks_riverpod 2.5.1 [collection flutter flutter_hooks flutter_riverpod riverpod state_notifier] - json_annotation 4.9.0 [meta] - pdf 3.11.0 [archive barcode bidi crypto image meta path_parsing vector_math xml] - printing 5.13.1 [ffi flutter flutter_web_plugins http image meta pdf pdf_widget_wrapper plugin_platform_interface web] - riverpod_annotation 2.3.5 [meta riverpod] - web 0.5.1 dev dependencies: - build_runner 2.4.11 [analyzer args async build build_config build_daemon build_resolvers build_runner_core code_builder collection crypto dart_style frontend_server_client glob graphs http_multi_server io js logging meta mime package_config path pool pub_semver pubspec_parse shelf shelf_web_socket stack_trace stream_transform timing watcher web_socket_channel yaml] - flutter_lints 3.0.2 [lints] - flutter_test 0.0.0 [flutter test_api matcher path fake_async clock stack_trace vector_math leak_tracker_flutter_testing async boolean_selector characters collection leak_tracker leak_tracker_testing material_color_utilities meta source_span stream_channel string_scanner term_glyph vm_service] - freezed 2.5.7 [analyzer build build_config collection meta source_gen freezed_annotation json_annotation dart_style] - json_serializable 6.8.0 [analyzer async build build_config collection json_annotation meta path pub_semver pubspec_parse source_gen source_helper] - riverpod_generator 2.4.2 [analyzer build build_config collection crypto meta path riverpod_analyzer_utils riverpod_annotation source_gen] transitive dependencies: - _fe_analyzer_shared 72.0.0 [meta] - _flutterfire_internals 1.3.39 [collection firebase_core firebase_core_platform_interface flutter meta] - _macros 0.3.2 - analyzer 6.7.0 [_fe_analyzer_shared collection convert crypto glob macros meta package_config path pub_semver source_span watcher yaml] - analyzer_plugin 0.11.3 [analyzer collection dart_style pub_semver yaml] - archive 3.6.1 [crypto path] - args 2.5.0 - async 2.11.0 [collection meta] - barcode 2.2.8 [meta qr] - bidi 2.0.10 - boolean_selector 2.1.1 [source_span string_scanner] - build 2.4.1 [analyzer async convert crypto glob logging meta package_config path] - build_config 1.1.1 [checked_yaml json_annotation path pubspec_parse yaml] - build_daemon 4.0.2 [built_collection built_value crypto http_multi_server logging path pool shelf shelf_web_socket stream_transform watcher web_socket_channel] - build_resolvers 2.4.2 [analyzer async build collection convert crypto graphs logging package_config path pool pub_semver stream_transform yaml] - build_runner_core 7.3.1 [async build build_config build_resolvers collection convert crypto glob graphs json_annotation logging meta package_config path pool timing watcher yaml] - built_collection 5.1.1 - built_value 8.9.2 [built_collection collection fixnum meta] - characters 1.3.0 - checked_yaml 2.0.3 [json_annotation source_span yaml] - clock 1.1.1 - code_builder 4.10.0 [built_collection built_value collection matcher meta] - collection 1.18.0 - convert 3.1.1 [typed_data] - cross_file 0.3.4+1 [meta web] - crypto 3.0.3 [typed_data] - custom_lint_core 0.6.3 [analyzer analyzer_plugin collection glob matcher meta package_config path pubspec_parse source_span yaml] - dart_style 2.3.6 [analyzer args collection path pub_semver source_span] - fake_async 1.3.1 [clock collection] - ffi 2.1.2 - file 7.0.0 [meta path] - firebase_app_check_platform_interface 0.1.0+33 [_flutterfire_internals firebase_core flutter meta plugin_platform_interface] - firebase_app_check_web 0.1.2+11 [_flutterfire_internals firebase_app_check_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins web] - firebase_auth_platform_interface 7.4.2 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface] - firebase_auth_web 5.12.4 [firebase_auth_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins http_parser meta web] - firebase_core_platform_interface 5.1.0 [collection flutter flutter_test meta plugin_platform_interface] - firebase_core_web 2.17.3 [firebase_core_platform_interface flutter flutter_web_plugins meta web] - firebase_storage_platform_interface 5.1.26 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface] - firebase_storage_web 3.9.11 [_flutterfire_internals async firebase_core firebase_core_web firebase_storage_platform_interface flutter flutter_web_plugins http meta web] - fixnum 1.1.0 - flutter_plugin_android_lifecycle 2.0.21 [flutter] - flutter_riverpod 2.5.1 [collection flutter meta riverpod state_notifier] - flutter_web_plugins 0.0.0 [flutter characters collection material_color_utilities meta vector_math] - frontend_server_client 4.0.0 [async path] - glob 2.1.2 [async collection file path string_scanner] - google_generative_ai 0.4.3 [http] - graphs 2.3.2 [collection] - http 1.2.2 [async http_parser meta web] - http_multi_server 3.2.1 [async] - http_parser 4.0.2 [collection source_span string_scanner typed_data] - image 4.2.0 [archive meta xml] - io 1.0.4 [meta path string_scanner] - js 0.7.1 - leak_tracker 10.0.5 [clock collection meta path vm_service] - leak_tracker_flutter_testing 3.0.5 [flutter leak_tracker leak_tracker_testing matcher meta] - leak_tracker_testing 3.0.1 [leak_tracker matcher meta] - lints 3.0.0 - logging 1.2.0 - macros 0.1.2-main.4 [_macros] - matcher 0.12.16+1 [async meta stack_trace term_glyph test_api] - material_color_utilities 0.11.1 [collection] - meta 1.15.0 - mime 1.0.5 - package_config 2.1.0 [path] - path 1.9.0 - path_parsing 1.0.1 [vector_math meta] - pdf_widget_wrapper 1.0.4 [flutter pdf] - petitparser 6.0.2 [meta] - plugin_platform_interface 2.1.8 [meta] - pool 1.5.1 [async stack_trace] - pub_semver 2.1.4 [collection meta] - pubspec_parse 1.3.0 [checked_yaml collection json_annotation pub_semver yaml] - qr 3.0.2 [meta] - riverpod 2.5.1 [collection meta stack_trace state_notifier] - riverpod_analyzer_utils 0.5.3 [analyzer collection crypto custom_lint_core freezed_annotation meta path source_span] - shelf 1.4.1 [async collection http_parser path stack_trace stream_channel] - shelf_web_socket 2.0.0 [shelf stream_channel web_socket_channel] - sky_engine 0.0.99 - source_gen 1.5.0 [analyzer async build dart_style glob path source_span yaml] - source_helper 1.3.4 [analyzer collection source_gen] - source_span 1.10.0 [collection path term_glyph] - stack_trace 1.11.1 [path] - state_notifier 1.0.0 [meta] - stream_channel 2.1.2 [async] - stream_transform 2.1.0 - string_scanner 1.3.0 [source_span] - term_glyph 1.2.1 - test_api 0.7.3 [async boolean_selector collection meta source_span stack_trace stream_channel string_scanner term_glyph] - timing 1.0.1 [json_annotation] - typed_data 1.3.2 [collection] - vector_math 2.1.4 - vm_service 14.2.4 - watcher 1.1.0 [async path] - web_socket 0.1.6 [web] - web_socket_channel 3.0.1 [async crypto stream_channel web web_socket] - win32 5.5.1 [ffi] - xml 6.5.0 [collection meta petitparser] - yaml 3.1.2 [collection source_span string_scanner] ```

Additional context and comments

Firebase App Check debug token also does not work.

Lyokone commented 4 months ago

Hello @garysm, You can try to use

FirebaseVertexAI instanceFor({
    FirebaseApp? app,
    FirebaseAppCheck? appCheck,
    FirebaseAuth? auth,
    RequestOptions? options,
    String? location,
  })

and provide it with the correct appCheck instance?

garysm commented 4 months ago

Hello @garysm, You can try to use
FirebaseVertexAI instanceFor({
    FirebaseApp? app,
    FirebaseAppCheck? appCheck,
    FirebaseAuth? auth,
    RequestOptions? options,
    String? location,
  })
and provide it with the correct appCheck instance?

Strange, that actually did work when working with the App Check debug token, but doesn't when deployed to Firebase Hosting.

garysm commented 4 months ago

Looks like this fix worked after some time. Is there some way the App Check instance can be detected without declaring it ourselves?

firebase / flutterfire