AppFlowy Testing
 |
AppFlowy is an open-source alternative to Notion, designed as an AI-powered workspace where users can organize knowledge, manage projects, and collaborate—all while maintaining full control of their data. Built with Flutter for the frontend and Rust for the backend, AppFlowy offers cross-platform compatibility across web, desktop (macOS, Windows, Linux), and mobile (iOS, Android).
Testing a productivity application like AppFlowy requires validating complex user workflows—document creation, workspace management, real-time collaboration, and Kanban boards. End-to-end testing is the most effective way to ensure these features work correctly from the user's perspective.
End-to-End Testing with testRigor
testRigor specializes in end-to-end testing and is particularly well-suited for testing AppFlowy's web application. Among the key features are ease of test creation, unrivaled test stability, and low maintenance—a rare combination for E2E tests.
- Test complex document editing workflows including slash commands and rich text formatting
- Validate workspace management, including creation, renaming, and deletion
- Test real-time collaboration features and user invitations
- Verify Kanban board operations, card management, and filtering
- Support email testing for invitation workflows and account verification
- Support visual testing if needed (e.g., compare document layouts across deployments)
End-to-end tests in testRigor represent how a real user would interact with AppFlowy. What this means is that you don't need to use CSS Selectors or dig into the DOM structure to define an element. Commands are in simple English terms, such as click "New page" or check that page contains "Getting started".
Step-by-Step: Document Lifecycle Test Case
Let's walk through a complete end-to-end test that covers the entire document lifecycle in AppFlowy: create, edit, delete, restore, and publish. This test demonstrates the power and simplicity of testRigor's plain-English commands.
Part 1: Login and Create a Document
First, we log in using a reusable rule, then create a new document with a unique title:
// Use the reusable login rule
Login using password
check that page contains "New page"
// Create a new document
click "New page"
check that page contains "Where would you like to add"
click "General"
click "Add"
check that page contains "Enter a / to insert a block, or start typing"
check that page contains "Untitled"
// Set a unique title (triple-click selects all text)
triple click "Untitled"
generate from regex "dq[a-z]{5}" and save as "newTitle"
type stored value "newTitle"
type enter
check that page contains stored value "newTitle"
// Add body content
enter enter
type "QA body text"
type enter
check that page contains "QA body text"
// Expand to full page view
click on "the expand (open as page) icon at the top-left of the document dialog" using ai
check that page contains stored value "newTitle"
check that page contains "Share"
Part 2: Delete the Document
Delete the document using the more options menu:
// Open the more options menu (full-page view: Share | ★ | ⋯) click on "the more options (...) button at the top-right of the page, to the right of the star favorite icon" using ai check that page contains "Delete" click "Delete" click "Delete" to the right of "Cancel" if exists // Verify deletion check that page does not contain stored value "newTitle"
Part 3: Restore from Trash
Navigate to the trash and restore the deleted document:
// Open the trash click on "the trash bin icon at the bottom-left of the sidebar" using ai check that page contains "Restore All" check that page contains "Delete All" // Find and restore the document check that page contains stored value "newTitle" below "File name" hover over stored value "newTitle" below "File name" click on "the Restore icon (counterclockwise arrow) on the same row as the stored file name in the trash list, not Restore All and not the row delete/trash icon" using ai // Verify removal from trash check that page does not contain stored value "newTitle" below "File name" // Navigate back to the restored document click "Shared" check that page contains "Shared" hover over "General" click 2nd button to the left of "General" check that page contains stored value "newTitle" click stored value "newTitle"
Part 4: Publish to Web
Publish the document as a public website and verify the published page:
// Verify we're on the document page check that page contains "My Workspace" check that page does not contain "Comments" // Publish the document click "Share" click "Publish" click "Publish" roughly below "Create a website with AppFlowy" click "Visit site" // Verify the published page check that page contains stored value "newTitle" check that page does not contain "My Workspace" scroll down until page contains "Comments" // Test adding a comment on the published page generate unique name, then enter into "Add a comment" and save as "newComment" click "Add" check that page contains saved value "newComment" roughly below "Comments" // Test the Comments toggle (disable comments) switch to tab 1 enter "0" into input to the right of "Comments" switch to tab 2 reload check that page contains stored value "newTitle" check that page does not contain "Comments" // Test the Template toggle (disable template option) check that page contains "Start with this template" switch to tab 1 enter "0" into input to the right of "Duplicate as template" switch to tab 2 reload check that page contains stored value "newTitle" check that page does not contain "Start with this template"
Part 5: Clean Up
Delete the document to leave the workspace clean for future tests:
// Switch back to the editor tab switch to tab 1 type escape check that page contains "Share" // Delete the document permanently click on "the more options (...) button at the top-right of the page, to the right of the star favorite icon" using ai check that page contains "Delete" click "Delete" click "Delete" to the right of "Cancel" if exists check that page does not contain stored value "newTitle"
This single test case validates five critical user journeys: document creation, content editing, deletion, restoration from trash, and web publishing—all in plain English that any team member can read and understand.
Running AppFlowy Tests with testRigor
To make it easier to get started with AppFlowy testing, we've created a set of pre-built test cases that cover critical AppFlowy workflows. These tests are available as individual public execution links below, where you can view and copy each test case to your own account.
Quick Setup Guide
Follow these simple steps to run the pre-built AppFlowy tests:
Step 1: Create Your testRigor Account
- Go to testRigor.com
- Click "Sign Up" and select "Public Open Source" version
- Complete registration and verify your email
- Log in to your account
Step 2: Access and Copy Individual Test Cases
Each test case has its own public execution link where you can view and copy it to your account. Click on each test case below to access it:
Pre-built Test Cases
The pre-built test suite includes these essential AppFlowy workflows:
1. Document Lifecycle (Create, Edit, Delete, Restore, Publish)
Test: Complete document workflow including creation, editing, deletion, trash restoration, and publishing to web
Why it's critical: Documents are the core of AppFlowy. This comprehensive test validates the entire document lifecycle—from creation through the slash menu, to editing content, deleting and restoring from trash, and finally publishing to a public website.
2. Rich Text Blocks & Slash Menu
Test: Insert various block types (headings, to-dos, quotes, callouts, code blocks) via the slash command menu
Why it's critical: The slash menu is AppFlowy's primary interface for content creation. This test ensures all block types render correctly and that the slash command system works reliably.
3. User Registration & Workspace Management
Test: Sign up for a new account, create a workspace, rename it, and clean up by deleting the account
Why it's critical: User onboarding is the first experience new users have with AppFlowy. This test validates the complete registration flow, workspace creation, and workspace management.
4. Kanban Board Full Regression
Test: Complete Kanban board testing including card operations (add, edit, duplicate, delete), group management (add, rename, hide, delete), and field type changes
Why it's critical: Kanban boards are essential for project management in AppFlowy. This comprehensive test covers all card operations, column management, and the interaction between cards and their status fields.
5. Workspace Invitation & Collaboration
Test: Invite a new user to a workspace and verify the invitation acceptance flow
Why it's critical: Collaboration is a core feature of AppFlowy. This test validates that workspace owners can invite team members and that invitees can successfully join the workspace.
6. Favorites & Sub-pages Navigation
Test: Add pages to favorites, create sub-pages (nested documents), and verify navigation between them
Why it's critical: As workspaces grow, users rely on favorites for quick access and sub-pages for organization. This test ensures the hierarchical page structure works correctly.
7. Global Search & Quick-find
Test: Use global search to find documents, verify search results accuracy, and test quick-find keyboard shortcuts
Why it's critical: With many documents in a workspace, search becomes the fastest way to navigate. This test validates that search returns accurate results and that keyboard shortcuts work as expected.
8. Grid Database with Sort & Filter
Test: Create a grid database, add rows, apply sorting and filtering operations
Why it's critical: Grid databases are AppFlowy's spreadsheet-like feature for structured data. Sorting and filtering allow users to manage large datasets effectively.
- Click one of the test case links above to open it
- Copy the test steps from the displayed test case
- Log in to your testRigor account and create a new test case
- Paste the copied steps into your new local test case
- Save the test case in your test suite
Step 3: Configure Your Test Suite
Set up your test suite with the AppFlowy Web URL:
- For AppFlowy Cloud:
https://appflowy.com - For Self-hosted: Your self-hosted AppFlowy URL (e.g.,
https://appflowy.yourcompany.com)
Step 4: Set Up Test Data Variables
Create the following global variables in your testRigor test suite:
username: A valid AppFlowy account emailpassword: The corresponding account password
These variables are used by the reusable "Login using password" rule that most test cases depend on.
Step 5: Run the Tests
From your testRigor dashboard:
- Select the test cases you want to run
- Choose your browser(s) - Chrome, Firefox, Safari, or Edge
- Click "Run" to execute the tests
Reusable Rules for AppFlowy Testing
testRigor's reusable rules allow you to create modular test components that can be shared across multiple test cases. For AppFlowy, we've created several commonly-used rules:
Login using password
A reusable rule that handles the login flow:
enter saved value "username" into "Please enter your email address" click "Continue with password" enter stored value "password" into "Password" click "Continue" check that page contains "New Page"
Sign up using password
A reusable rule for creating new test accounts with unique credentials for isolation.
Delete account
A cleanup rule that removes test accounts after test execution to maintain a clean test environment.
Testing AppFlowy-Specific Features
AppFlowy has unique features that require specific testing approaches. Here are real examples from our test suite:
Rich Text Blocks & Slash Menu
The slash command menu is AppFlowy's primary interface for inserting content blocks. Here's how we test inserting headings, to-dos, quotes, and code blocks:
// Insert a heading via slash menu type "/heading" type enter type "Heading One" type enter check that page contains "Heading One" // Insert a to-do item type "/todo" type enter type "Todo item one" type enter check that page contains "Todo item one" // Insert a quote block type "/quote" type enter type "Quote text here" type enter check that page contains "Quote text here" // Insert a code block type "/code" type enter type "codeword123" check that page contains "codeword123"
Global Search & Quick-find
Testing AppFlowy's search functionality involves verifying both empty states and successful searches:
// Test empty-state search
click "Search"
check that page contains "Recent pages"
enter "zzqqxxnomatchquery" into input
check that page does not contain "Recent pages"
check that page contains "No search results"
check that page contains "Ask AI"
type escape
// Create a page, then search for it
click "New page"
click "General"
click "Add"
triple click "Untitled"
generate from regex "sq[a-z]{5}" and save as "newTitle"
type stored value "newTitle"
type enter
check that page contains stored value "newTitle"
// Search for the page just created
click "Search"
enter stored value "newTitle" into input
check that page does not contain "No search results"
check that page contains stored value "newTitle"
Favorites & Sub-pages
Testing page organization features like favorites and nested sub-pages:
// Favorite a page
click on "the star (favorite) icon in the top-right toolbar of the open page" using ai
check that page contains "Favorites"
check that page contains stored value "newTitle" roughly below "Favorites"
// Unfavorite and verify tooltip
click on "the star (favorite) icon in the top-right toolbar of the open page" using ai
wait 1 second
hover over "the star (favorite) icon in the top-right toolbar of the open page" using ai
check that page contains "Add to Favorites"
// Create sub-page via slash menu
enter "/" into "Enter a / to insert a block, or start typing"
type "sub page"
click "Document"
generate from regex "fc[a-z]{5}", then enter into "Title" and save as "childTitle"
check that page contains stored value "childTitle"
// Verify nesting via breadcrumb
check that page contains stored value "newTitle" to the left of stored value "childTitle"
AI-Assisted Interactions
Some AppFlowy UI elements require AI-powered element identification for reliable testing:
// Open document as full page click on "the expand (open as page) icon at the top-left of the document dialog" using ai // Access more options menu click on "the more options (...) button at the top-right of the page, to the right of the star favorite icon" using ai // Access trash click on "the trash bin icon at the bottom-left of the sidebar" using ai // Restore from trash click on "the Restore icon (counterclockwise arrow) on the same row as the stored file name in the trash list" using ai
Dynamic Content & Workspace Management
AppFlowy tests often need to create unique content and manage workspaces:
// Generate unique page titles to avoid conflicts
generate from regex "dq[a-z]{5}" and save as "newTitle"
type stored value "newTitle"
check that page contains stored value "newTitle"
// Create and rename workspace
click "My Workspace"
click "Create workspace"
generate unique name, then enter into "name" and save as "workspaceName"
click "Create"
check that page contains saved value "workspaceName"
// Rename workspace
click 1st button to the right of 2nd saved value "workspaceName"
click "Rename workspace"
generate unique name, then enter into "name" and save as "newWorkspaceName"
click "Rename"
check that page contains saved value "newWorkspaceName"
Setting Up GitHub CI/CD for testRigor
Integrating testRigor tests into your GitHub CI/CD pipeline ensures that every code change is validated against your end-to-end tests before deployment.
Step 1: Get Your testRigor Credentials
- Log in to your testRigor account
- Navigate to your test suite settings
- Copy your Test Suite ID (found in the URL or settings page)
- Generate an API Auth Token from your account settings
Step 2: Add GitHub Secrets and Variables
- Go to your GitHub repository
- Click on Settings → Secrets and variables → Actions
- Add the following:
As a Secret (keeps it encrypted):
TESTRIGOR_TOKEN: Your testRigor API authentication token
As Variables:
TESTRIGOR_SUITE_ID: Your testRigor test suite IDAPPFLOWY_BASE_URL: Your AppFlowy instance URL (e.g.,https://appflowy.com)
For further information on creating secrets and variables in GitHub Actions, check the official GitHub documentation on secrets and variables.
Step 3: Create the GitHub Actions Workflow
Create a new file at .github/workflows/testrigor.yml in your repository:
name: AppFlowy E2E Tests with testRigor
on:
push:
branches: [main]
pull_request:
branches: [main]
# Allow manual trigger
workflow_dispatch:
jobs:
e2e-tests:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install testRigor CLI
run: npm install -g testrigor
- name: Run testRigor Tests
run: |
testrigor run \
--suite ${{ vars.TESTRIGOR_SUITE_ID }} \
--token ${{ secrets.TESTRIGOR_TOKEN }} \
--base-url ${{ vars.APPFLOWY_BASE_URL }} \
--wait \
--junit-report results.xml
- name: Upload Test Results
uses: actions/upload-artifact@v4
if: always()
with:
name: testrigor-results
path: results.xml
- name: Publish Test Results
uses: dorny/test-reporter@v1
if: always()
with:
name: testRigor E2E Results
path: results.xml
reporter: java-junit
Step 4: Configure Test Execution Options
The testRigor CLI supports several options to customize your CI runs:
| Option | Description |
|---|---|
--suite |
Test suite ID to run |
--token |
API authentication token |
--base-url |
Override the test suite's base URL |
--wait |
Wait for tests to complete before exiting |
--junit-report |
Generate JUnit XML report for CI integration |
--labels |
Run only tests with specific labels (e.g., --labels smoke,critical) |
--parallel |
Number of parallel test executions |
Step 5: Running Tests on Pull Requests
For pull request validation, you may want to run a subset of tests (smoke tests) to get faster feedback:
name: PR Smoke Tests
on:
pull_request:
branches: [main]
jobs:
smoke-tests:
runs-on: ubuntu-latest
steps:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install testRigor CLI
run: npm install -g testrigor
- name: Run Smoke Tests
run: |
testrigor run \
--suite ${{ vars.TESTRIGOR_SUITE_ID }} \
--token ${{ secrets.TESTRIGOR_TOKEN }} \
--labels smoke \
--wait
Step 6: Scheduled Regression Testing
Run full regression tests on a schedule (e.g., nightly):
name: Nightly Regression Tests
on:
schedule:
# Run at 2 AM UTC every day
- cron: '0 2 * * *'
workflow_dispatch:
jobs:
regression-tests:
runs-on: ubuntu-latest
steps:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install testRigor CLI
run: npm install -g testrigor
- name: Run Full Regression
run: |
testrigor run \
--suite ${{ vars.TESTRIGOR_SUITE_ID }} \
--token ${{ secrets.TESTRIGOR_TOKEN }} \
--wait \
--junit-report results.xml
- name: Upload Test Results
uses: actions/upload-artifact@v4
if: always()
with:
name: regression-results
path: results.xml
Viewing Test Results
After the workflow runs:
- Go to the Actions tab in your GitHub repository
- Click on the workflow run to see the summary
- Download the JUnit report artifact for detailed results
- View the test reporter output for pass/fail breakdown
- Click the testRigor dashboard link in the logs for screenshots and video recordings
Troubleshooting CI/CD Issues
- Tests timing out: Increase the timeout in your workflow or check if the AppFlowy instance is accessible from GitHub runners
- Authentication failures: Verify your
TESTRIGOR_TOKENsecret is correctly set and hasn't expired - Base URL issues: Ensure your AppFlowy instance is publicly accessible or use a tunnel for private instances
- Flaky tests: Use testRigor's built-in retry mechanism and AI-powered selectors to improve stability
