Adding A/B Testing to Your Hydrogen Storefront

This guide will walk you through setting up Pack's A/B testing feature on your Hydrogen storefront to deliver flicker-free test experiences with clean data integration.

Prerequisites

Before implementing Pack's A/B testing, ensure your data infrastructure is ready:

Required Data Infrastructure:

• BigQuery connection (BigQuery, Snowflake, Databricks, etc.)
• Clean data collection platform (Elevar, Fueled, Blotout, Littledata, or similar)
• Server-side event tracking with proper attribution
• Analytics governance and established KPI definitions

Technical Requirements:

• Hydrogen storefront running on Shopify
• Access to your storefront's codebase
• BigQuery connection for test reporting
• Analytics platform integration (GA4, etc.)

1. Update Packages

Update the following packages to their latest ab-test versions:

• @pack/hydrogen
• @pack/react

npm install @pack/hydrogen@1.0.6-ab-test.11
npm install @pack/react@0.1.4-ab-test.1

2. Update server.ts and add testSession to createPackClient

  import {createPackClient, PackSession, PackTestSession, handleRequest} from '@pack/hydrogen'; // Import PackTestSession
  // other imports and code...
  const [cache, session, packSession, testSession] = await Promise.all([ // Add testSession
    caches.open('hydrogen'),
    AppSession.init(request, [env.SESSION_SECRET]),
    PackSession.init(request, [env.SESSION_SECRET]),
    PackTestSession.init(request, [env.SESSION_SECRET]), // Add this line
  ]);

  const pack = createPackClient({
    cache,
    waitUntil,
    token: env.PACK_SECRET_TOKEN,
    storeId: env.PACK_STOREFRONT_ID,
    session: packSession,
    contentEnvironment: env.PACK_CONTENT_ENVIRONMENT,
    defaultThemeData,
    testSession, // Add this line
  });

3. Add PackTestProvider to layout.tsx

Integrate PackTestProvider to manage A/B test exposure.

Import PackTestProvider

Add this import at the top of layout.tsx:

Retrieve Consent & Test Exposure Handler

Use the useTestExpose hook to obtain hasUserConsent and handleTestExpose:

const { handleTestExpose, hasUserConsent } = useTestExpose();

Wrap the Storefront in PackTestProvider

Wrap your main layout inside PackTestProvider:

<>
  <Analytics />
  <PackTestProvider hasUserConsent={hasUserConsent} testExposureCallback={handleTestExpose}>
    <div>
      <Header />

      <main>
        {children}
      </main>

      <Footer />

      <Cart />

      <Search />

      <Modal />
    </div>
  </PackTestProvider>
</>

4. Implement the useTestExpose hook in Layout.tsx

The useTestExpose hook manages user consent for A/B testing and sends exposure events to analytics.

View the full implementation on GitHub

Usage Example

import { useTestExpose } from '~/hooks/useTestExpose';

export function Layout({children}: {children: ReactNode}) {

  const { handleTestExpose, hasUserConsent } = useTestExpose();

    return (
    <>
      <PackTestProvider
        hasUserConsent={hasUserConsent}
        testExposureCallback={handleTestExpose}
      >
        <div>
          <Header />
          <main>{children}</main>
          <Footer />
          <Cart />
          <Search />
          <Modal />
        </div>
      </PackTestProvider>
    </>
  );
};

Features

• ✅ Publishes EXPERIMENT_EXPOSED events when consent is granted.
• ✅ Pushes experiment exposure data to window.dataLayer for GTM tracking.
• ✅ Uses a useRef to maintain the latest publish function from useAnalytics.

API

handleTestExpose(test: any)

Fires an experiment exposure event only if consent is granted.

• Parameters:
  • test.handle (string) – Experiment name.
  • test.testVariant.handle (string) – Active variation.

hasUserConsent: boolean

Indicates whether the user has consented to functional cookies and analytics.

5. Add PackTestRoute to All Relevant Storefront Routes

Integrate PackTestRoute to register test-related routes in your storefront (e.g., ($locale)._index.tsx, ($locale).articles.$handle.tsx, ($locale).blogs.$handle.tsx, ($locale).collections.$handle.tsx, ($locale).pages.$handle.tsx, ($locale).products.$handle.tsx).

Import PackTestRoute Component

import { PackTestRoutes } from '@pack/hydrogen';

Add PackTestRoute to Your Routes

For example, in app/routes/($locale)._index.tsx:

import { useLoaderData } from '@remix-run/react';
import { json } from '@shopify/remix-oxygen';
import type { LoaderFunctionArgs } from '@shopify/remix-oxygen';
import { RenderSections } from '@pack/react';
import { PackTestRoute } from '@pack/hydrogen';

import { PAGE_QUERY } from '~/data/queries';

export async function loader({ context, params, request }: LoaderFunctionArgs) {
  const { data, packTestInfo } = await context.pack.query(PAGE_QUERY, {
    variables: { handle: '/' },
    cache: context.storefront.CacheLong(),
  });

  if (!data?.page) throw new Response(null, { status: 404 });

  return json({
    page: data.page,
    url: request.url,
    packTestInfo,
  });
}

export default function Index() {
  const { page } = useLoaderData<typeof loader>();

  return (
    <>
      <PackTestRoute />
      <RenderSections content={page} />;
    </>
  );
}

6. Update Analytics Component and Relevant Integrations

The original PackAnalytics component is now renamed to Analytics.

Add EXPERIMENT_EXPOSED to Analytics Constants

Update app/components/Analytics/constants.ts:

export const AnalyticsEvent = {
  ...HydrogenAnalyticsEvent,
  PRODUCT_VARIANT_SELECTED: 'custom_product_variant_selected',
  PRODUCT_ITEM_CLICKED: 'custom_product_item_clicked',
  PRODUCT_QUICK_SHOP_VIEWED: 'custom_product_quick_shop_viewed',
  CUSTOMER: 'custom_customer',
  CUSTOMER_SUBSCRIBED: 'custom_customer_subscribed',
  CUSTOMER_LOGGED_IN: 'custom_customer_logged_in',
  CUSTOMER_REGISTERED: 'custom_customer_registered',
  // New event for A/B testing experiment exposure
  EXPERIMENT_EXPOSED: 'custom_experiment_exposed',
} as typeof HydrogenAnalyticsEvent & {
  PRODUCT_VARIANT_SELECTED: 'custom_product_variant_selected';
  PRODUCT_ITEM_CLICKED: 'custom_product_item_clicked';
  PRODUCT_QUICK_SHOP_VIEWED: 'custom_product_quick_shop_viewed';
  CUSTOMER: 'custom_customer';
  CUSTOMER_SUBSCRIBED: 'custom_customer_subscribed';
  CUSTOMER_LOGGED_IN: 'custom_customer_logged_in';
  CUSTOMER_REGISTERED: 'custom_customer_registered';
  EXPERIMENT_EXPOSED: 'custom_experiment_exposed';
};

Add experimentExposedEvent to Integrations

For example, in app/components/Analytics/GA4Events:

// app/components/Analytics/GA4Events/events.ts

const experimentExposedEvent = ({
  debug,
  ...data
}: Record<string, any> & { debug?: boolean }) => {
  const analyticsEvent = AnalyticsEvent.EXPERIMENT_EXPOSED;

  try {
    if (debug) logSubscription({ data, analyticsEvent });

    const { test, customer } = data;

    if (!test) throw new Error('`test` parameter is missing.');

    const event = {
      event: 'view_experiment',
      user_properties: generateUserProperties({ customer }),
      experiment_id: test?.id,
      experiment_name: test?.handle,
      experiment_variant_id: test?.testVariant.id,
      experiment_variation: test?.testVariant.handle,
    };

    emitEvent({ event, debug });
  } catch (error) {
    logError({
      analyticsEvent,
      message: error instanceof Error ? error.message : error,
    });
  }
};

export {
  experimentExposedEvent,
  // ... additional exports
};

And in app/components/Analytics/GA4Events/GA4Events.tsx:

import { useEffect, useState } from 'react';

import { AnalyticsEvent } from '../constants';
import {
  experimentExposedEvent, // newly added
  ANALYTICS_NAME,
} from './events';

export function GA4Events({ ga4TagId, register, subscribe, customer, debug = false }) {
  // ... initialization and script loading logic

  useEffect(() => {
    if (!scriptLoaded) return;

    subscribe(AnalyticsEvent.EXPERIMENT_EXPOSED, (data: Data) => {
      experimentExposedEvent({ ...data, customer, debug });
    });

    // ... additional subscriptions
  }, [scriptLoaded, customer?.id, debug]);

  // ... rest of the code
}

7. Verification Steps

1. Run the storefront locally.
2. Visit an A/B test URL (e.g., /test-xyz) with an active test set up in Pack Admin – A/B Tests.
3. Confirm that handleTestExpose is firing. You should see the console log:

   ==== TEST EXPOSED ====
   

4. Check GTM Debug Mode and GA4 DebugView to validate event tracking.

Consent Considerations

You can refer to the Consent Management documentation for more details on how to manage user consent effectively.

GA4 / GTM Integration

Please refer to GTM & GA4 Integration.

QA & Testing Plan

QA Test Cases

Functional Cookie Consent Testing:

Event Debugging & Validation

GTM Event Validation

GA4 Debugging & Validation

BigQuery Data Validation

Debugging Tools

• Console Log: Look for the ==== TEST EXPOSED ==== message.
• GTM Debug Mode: Append ?gtm_debug=x to the URL.
• GA4 DebugView: Accessible via Admin > DebugView in GA4.
• Chrome DevTools: Inspect Console > Network > Analytics Requests.
• BigQuery: Run queries like:

  SELECT * FROM analytics_GA4PROPERTYID.events_* ORDER BY event_timestamp DESC
  

Reporting & Fixing Issues

1. Verify that hasUserConsent is true.
2. Ensure handleTestExpose is triggered correctly.
3. Confirm GTM receives the view_experiment event.
4. Check that GA4 logs the event.
5. Debug in BigQuery if GA4 logs but data isn’t received.

Summary

• ✅ ==== TEST EXPOSED ==== appears when an experiment is correctly logged.
• ✅ GTM Preview Mode shows view_experiment when consent is granted.
• ✅ GA4 DebugView confirms event logging.
• ✅ BigQuery records event data daily.

QA OPEN ITEM

• Testing cookie consent in preview environments is challenging.
  • Shopify’s consent banner may only fire in production.
  • OneTrust requires a permanent staging URL and test scripts for QA before production.

Troubleshooting

Common Issues:

Test events missing in BigQuery:

• Verify GA4 is tracking events on your storefront (check GA4 real-time reports)
• Confirm GA4 → BigQuery linking is enabled in GA4 Admin
• Double-check BigQuery credentials in your Pack dashboard
• Wait 24-48 hours for data to appear (BigQuery export takes time)

Inconsistent test assignment:

• Pack components not implemented correctly in your React code
• Conflicting A/B testing code running alongside Pack
• User identification issues across sessions
• Outdated/incorrect @pack/react @pack/hydrogen packages

Data discrepancies between platforms:

• Make sure events aren't being sent twice (client + server)
• Align attribution windows (GA4 vs Shopify vs Other data sources)
• Pick one platform as your "source of truth" for each metric

Performance issues:

• Ensure A/B test logic runs server-side
• Avoid client-side variant switching (causes layout shifts)
• Check for unnecessary JavaScript loading that you may have added for the test

Getting Help:

Pack handles experience delivery, while your data partners handle measurement accuracy. For:

• Experience delivery issues: Contact Pack support
• Data discrepancies or attribution issues: Work with your analytics provider (Elevar, Fueled, etc.)
• BigQuery connection problems: Check with your data infrastructure team

Best Practices

1. Clean Event Tracking: Ensure consistent event naming and data structure
2. Proper Fallbacks: Always include fallback experiences for error cases
3. Performance Monitoring: Track test impact on page load times
4. Data Validation: Regularly verify data accuracy across all platforms

Next Steps

Once your A/B testing is set up:

1. Start with simple tests to validate your data flow
2. Run A/A tests to establish baseline performance
3. Scale gradually to more complex test experiences
4. Document learnings and share insights across your team

Need help with data infrastructure setup or connecting with analytics partners? Contact our team.

Remember: Pack delivers the experience, your analytics stack measures the results. This separation ensures you get both great user experiences and reliable data.