Ever since Flutter had its first stable version released, it has been growing rapidly among us mobile developers. Today, flutter is the default choice for many of us. However, the framework is still relatively new, and there are some aspects in which it is not very mature yet. Until December 2020, the framework didn't have a good solution for integration testing. Developers that wanted to go down this road needed to use flutter driver tests, which was not a robust tool and would not be enough to fully test a complex app, in my point of view.
The integration_test package
In December 2020, Flutter launched the package integration_test, which adapts the package flutter_test (used before for widgets testing and unit tests only) to integration testing. This brought us a lot of benefits and made it possible to write robust code to test the app as a whole. However it still has some drawbacks, and this article will explore the best ways in my experience to write robust integration tests code to get your app fully tested and make your customers happy :D
What are integration tests and why are they so important?
While unit tests make sure functions work individually and widget tests make sure individual widgets work as expected, integration testing combines everything and makes sure your entire app works as a whole. It simulates the user interaction navigating between different screens, interacting with buttons, etc, therefore making the tests elaborated enough to catch specific problems that can only happen within the app flow, and not isolated on a widget or function.
Setting up your integration tests
Disclaimer: This initial setup part (and more) can be found on the official flutter page for integration tests, which I recommend that you read, as it contains some information I won't mention here.
- Add the necessary packages to your pubspec.yaml file
2. Create a new directory app/test_driver, containing a file called integration_test.dart, with the content:
3. Every test file you create needs to be under the directory app/integration_test/, so create the directory and create a file to write your first test. The final directory structure should be similar to this:
Your test file should have this basic structure
4. Running your integration tests
The recommended way to run your integration tests according to the official flutter integration_test package page is:
This is the default way that should be used when you do integrate your tests with maybe Firebase Labs or GitHub Actions, however, it has a problem. While developing your tests, you will need to fix the code several times and re-run it. Running the command as above takes some starting time and it delays the test development. So a good solution is to run them (for development only) like this:
It gives you the ability to press uppercase "R" and hot restart your tests. This is something not mentioned on the official pages that helped me accelerate my integration testing development speed by a lot.
How to run all your tests at once
One "problem" with the new integration_test package, is that there isn't a built-in way to run all the tests at app/integration_test at once, so we need to set this up ourselves.
So, first of all, change your test files base structure to be like this (there are many ways to do it, this is just one example):
Now you can run your tests from the testFile1.dart file just by calling doTests1(). Therefore, you can create a file integration_test/app_test.dart that will call all the tests at once:
And finally, you can run all of them with one of the two commands below:
Common flutter_test methods
To give you a few examples of methods we can use, nothing better than an actual test code from a personal project I've been working on. This is available here.
Common problems and solutions
- Tests take too long to start and fail
Undoubtedly this was the issue I went through take took me the longest to figure out and once I did the tests started running smoothly. Adding await pump(), await pumpAndSettle(), or await Future<void>.delayed() were not nearly enough to get my tests to start consistently without failing. I read the source code for the pumpAndSettle() method and modified it so we can wait until a condition has been reached:
To use the waitUntil() function for your benefit, you can call it after await app.main(); and wait until one widget visible on your initial screen is displayed. This way, the test won't get stuck on the "Tests Starting" page and will start as soon as the test start-up is completed.
2. Third-party components don’t have keys, so I can’t code the tests to interact with their components?
It might be harder, but yes, you can. For this, you will mostly use finders ancestors and descendants. I suggest you make use of Android Studio Flutter Widgets Inspector and use it to find the widget you are looking for.
For the date picker component, for example, this is one of the infinity ways you can find the previous and next month buttons
3. "My finder is finding many widgets but only one is visible”
I've bumped into this problem a few times when using third-party components. Every finder has a property skipOffstage that is set to "true" by default, which means it won't find widgets that are off stage. However, even with it set to "true" I have experienced weird behaviors. When it happens, I suggest that you try being very specific with your finder, by using find.descendant() and/or find.ancestors() in a way that you can point out the exact widgets you are looking for.
4. When dealing with long animations or code delays (e.g. "await Future.delayed(Duration(seconds: 1))"), the "pumpAndSettle()" method is not enough to wait for it to finish.
To solve this, I recommend that you make use of the method waitUntil() (available on this page). You can simply wait for a condition that will only happen once the animation/delay/API call is finished.
5. My tests are taking very long to run, as it needs to wait for all the API calls
Well, there isn't any magic trick you can do to make all the API calls finishes instantly. The one thing you can do is to use a package such as Mockito to mock your API calls and then have your tests run without any significant delays.
Although this is a great way to optimize your tests and tests development speed, it's important to highlight that this is a trade-off, as you will be limiting your tests to test only the app code by itself, and not take into consideration API issues or even app wrong behaviors that would happen only with specific API responses that you may not think of when mocking the requests.
My recommended integration tests files structure
In my point of view, it is important that you have a structure that allows you to run all the tests at once (to be quicker with CI/CD) or individually (to be quicker to code the tests). This is explored in the "How to run all your tests at once" section above. Besides that, I think it's a good idea to have a file with common methods that all test files will use and one finders class, to isolate all the finders somewhere easy to change and that other tests may use as well.
We can easily use the common methods anywhere we want
You are free to organize your tests any way you want, this is just my recommendation based on my experience with integration tests developments for Flutter.
I hope this was useful to you. If you liked it, please leave a 👏 to incentivize me to write more guides like this.
Any questions or suggestions? Please leave a comment
Thank you for reading this!