Development

Validating data with JSON Schema, Angular and TypeScript

 Published: Sep 18, 2019  (last updated: Feb 12, 2021)~ 1900 words~ 9 minutes reading time

One common question I see with a lot of new TypeScript developers is how to handle runtime validation of data, using the types they have built.

The issue is web platform, as yet, does not support types. Typescript itself is a higher-level language built on top of JavaScript and uses a compiler to create compatible code for the web, node or other JS platforms - this means that types are only available at design time.

Most developers have a method or form in their code where they want to validate data being passed in is correct before sending it to another API. This works for hard coded data in Typescript, but not dynamic data from sources such as a form or API source

The good news is the problem itself has been solved and there are several solutions for TypeScript such as io-ts or joi but I find these solutions to encourage duplication of types across different domains to maintain both your types and validation objects.

Introducing JSON Schema

A much simpler way to maintain both types and validation within a project is to use a single source of truth. The main option for this is JSON Schema .

A JSON Schema file allows you to define types using a JSON file, using a specification defined by the selected draft (at the time of writing it’s number 7 ).

This file can be used to generate types for design-time coding using CLI tools and can be used for data validation at runtime using another library that can consume a schema to generate a validation method.

Schema Example

For this demo, I’ve created a simple schema object defining a customer within a system. The customers’ properties are:

  • ID
  • firstName
  • lastName
  • dateOfBirth
  • email

In this example we set "additionalProperties": false to keep the example simple, but it’s a very flexible option!

If set to true or not included, the outputted types will include an indexable type with a [key: string]: any at the end of the type properties.

You can also pass it properties such as "additionalProperties": { "type": "string" } which will allow only string additional properties to be added.

By setting to false - only the properties defined will be available on the type, which I’ll do for this example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
{
  "$id": "https://tane.dev/customer.json",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Customer Record",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "description": "The Customers ID in our system"
    },
    "firstName": {
      "type": "string",
      "description": "The customer's first name."
    },
    "lastName": {
      "type": "string",
      "description": "The customer's last name."
    },
    "email": {
      "type": "string",
      "format": "email",
      "description": "The customers email address"
    },
    "dateOfBirth": {
      "type": "string",
      "format": "date",
      "description": "The customer's date of birth."
    }
  },
  "additionalProperties": false,
  "required": [
    "id",
    "firstName",
    "lastName",
    "dateOfBirth",
    "email"
  ]
}

The first tool we will run this through is the imaginatively titled <code>json-schema-to-typescript</code> ! This project will take a valid schema file and generate a file containing the types. From the example above the output is:

json2ts -i customer.json -o customer.d.ts --style.singleQuote

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
/* tslint:disable */
/**
 * This file was automatically generated by json-schema-to-typescript.
 * DO NOT MODIFY IT BY HAND. Instead, modify the source JSONSchema file,
 * and run json-schema-to-typescript to regenerate this file.
 */

export interface CustomerRecord {
  /**
   * The Customers ID in our system
   */
  id: string;
  /**
   * The customer's first name.
   */
  firstName: string;
  /**
   * The customer's last name.
   */
  lastName: string;
  /**
   * The customers email address
   */
  email: string;
  /**
   * The customer's date of birth.
   */
  dateOfBirth: string;
}

One thing to note is that email and dateOfBirth are string in our type, the format is only used with validation. It is possible to create types for these fields and reference them using a more complex schema .

This type can now be imported into other types, and the json-schema-to-typescript will do this when you use complex references. For example, if we define an entire customer order type it might look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
import { CustomerRecord } from './customer';
import { OrderItem, Checkout, Address } from './shop-front'

export interface CustomerOrder {
  customer: CustomerRecord;
  deliveryAddress: Address;
  billingAddress: Address;
  items: OrderItem[]
  checkout: Checkout
}

Also, all the properties have been added to the required array. When creating a new customer, if the data does not contain an ID, you can use the Partial type to accept an incomplete object - if you expect your API to give back a full object you can return a CustomerRecord. You can also use Required where you need to ensure all fields are passed.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import { CustomerRecord } from './customer';

class CustomerClass {
  // Return a API request with a customers object
  async addCustomer(customer: Partial<CustomerRecord>): Promise<CustomerRecord> {
    return this.api.save(customer);
  }
  
  // Return a API request with a customers object
  async updateCustomer(customer: Required<CustomerRecord>): Promise<CustomerRecord> {
    return this.api.update(customer);
  }
}

Validating with the Schema

Now you have types, it makes the development of your application easier - but we still need to validate data entered is correct.

One way is to use the same schema on the server-side, using your languages JSON Schema validator, but in this example, I’ll use ajv - a javascript library that allows a schema to be loaded and data validated against it. The documentation is quite complete on using it in a JavaScript environment so I won’t repeat it too much here, but instead, I’ll build an Angular module that can be provided as a schema validation service.

First, we’ll create the Angular module, in to which we inject the AJV class and allow the user to provide a configuration, the service is provided below. This allows the module to be imported with a configuration and a service that is injectable through your application.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import { NgModule, InjectionToken } from '@angular/core';
import { HttpClientModule } from '@angular/common/http'
import { JSONSchemaService, AJV_INSTANCE } from './json-schema.service';
import ajv, { Ajv, Options } from 'ajv';

export const AJV_CLASS = new InjectionToken<Ajv>('The AJV Class Instance');
export const AJV_CONFIG = new InjectionToken<Ajv>('The AJV Class config');

export function createAjvInstance(AjvClass: any, config: Options) {
  return new AjvClass(config);
}

@NgModule({
  import: [HttpClientModule],
  provides: [
    JSONSchemaService,
    { provide: AJV_CLASS, useValue: ajv },
    { provide: AJV_CONFIG, useValue: {} },
    {
      provide: AJV_INSTANCE,
      useFactory: createAjvInstance,
      deps: [AJV_CLASS, AJV_CONFIG]
   }
  ]
})
export class JSONSchemaModule {}

Now we create a service - within this service it will access to the Ajv class that allows the service to be provided with schemas via an Angular HTTP call. The parsed schema is assigned a name and can be used through the app using dependency injection - this service is a good use case of a root service too, which create a Singleton of the service shared within the same application.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
import { Injectable, Inject, InjectionToken } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { Ajv } from 'ajv';

export const AJV_INSTANCE = new InjectionToken<Ajv>('The AJV Class Instance');

/**
 * The response of a validation result
 */
export interface ValidateResult {
  /**
   * If the result is valid or not
   */
  isValid: boolean;

  /**
   * Error text from the validator
   */
  errorsText: string;
}


@Injectable({
    provideIn: 'root'
})
export class JSONSchemaService {
  constructor(private readonly http: HttpClient, @Inject(AJV_INSTANCE) private readonly ajv: Ajv) {}

  /**
   * Fetches the Schema and adds it to the validator schema set
   * @param name The name of the schema, this will be used as the key to store it
   * @param urlPath The URL path of the schema to load
   */
  public loadSchema(name: string, urlPath: string): void {
    this.http.get(urlPath).subscribe(result => this.ajv.addSchema(result, name));
  }

  /**
   * Validate data against a schema
   * @param name The name of the schema to validate
   * @param data The data to validate
   */
  public validateData<T>(name: string, data: T): ValidateResult {
    const isValid = this.ajv.validate(name, data) as boolean;
    return { isValid, errorsText: this.ajv.errorsText() };
  }
}

Now we can use our service to load a JSON schemas into an internal Ajv map, and using the key load the schema to validate a data object against it. The service could be used alongside a form, any methods on a service or checking the result of one API before passing to another API.

A simple example of how it could be used in a form component (the example is shortened, most likely you would load your schemas from another service) or how you could validate the parameters passed to a method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
@Component({
  selector: 'my-form-component',
  template: `
    <errors-component *ngIf="let errors; errors$ | async"></errors-component>
    <form [formGroup]="customerForm" (ngSubmit)="submit()">
      <!-- Customer form in here --->
    </form>
  ` 
})
export class FormComponent {
  
  error$ = new BehaviorSubject<string>('');

  customerForm = this.fb.group({
    id: [''],
    firstName: [''],
    lastName: [''],
    email: [''],
    dateOfBirth: ['']
  });

  constructor(private readonly fb: FormBuilder, private readonly schema: JSONSchemaService, private readonly app: AppService) {
    this.schema.loadSchema('customer', 'https://tane.dev/customer.json')
  }

  /**
   * In the submit method, we validate the input of a form - this can be on top of, or instead
   * of Angular form validation
   */
  submit() {
    const result = this.schema.validateData('customer', this.customerForm.value);
    if (result.isValid) {
       this.app.updateCustomer(this.customerForm.value);
    } else {
      this.error$.next(result.errorsText);
    }
  }
 
  /**
   * This custom method can take a type of T (which in this case is an `any`) and validate
   * that the data is valid
   */
  customMethod<T = any>(data: T) {
    const result = this.schema.validateData('customer', data);
    if (result.isValid) {
       // Do custom logic
    } else {
      this.error$.next(result.errorsText);
    }
  }
}

Conclusion

I hope you’ve found this article useful in helping understand how and where Typescript can be used to validate data within an application, and JSON Schema to validate dynamic data.

Please feel free to leave feedback on any issues or improvements, but hopefully, these examples give a clearer understanding.

For full documentation of JSON Schema, check out the Understanding JSON Schema pages to get examples of using allOf, anyOf, oneOf and using definitions

Providing injectable features to Angular modules

 Published: Mar 14, 2019  (last updated: Mar 16, 2019)~ 1700 words~ 8 minutes reading time

Working with Angular and Typescript ; as I have refactored and re-written components I’ve been learning to take advantage of one of the more powerful features - dependency injection.

Building Angular components, most developers will have already used this to inject features like the HTTP client or the FormBuilder to be used in a component. A common service example might look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import { Injectable } from '@angular/core';
import { HttpClient, HttpResponse } from '@angular/common/http';
import { Observable } from 'rxjs';

@Injectable()
export class RequestService {
  constructor(private readonly http: HttpClient) {}

  public getExample(): Observable<HttpResponse> {
    return this.http.get('/api/example');
  }
}

Angular’s own features can be injected via their class identity (e.g. HttpClient). You can also inject any components, services or pipes that are registered within a module.

Angular also provides a mechanism for custom injections, this means items like configurations or instances of external libraries can be passed into any constructor within your module or any item within the dependency tree. To do this the library needs to provide an InjectionToken which represents the key in the dependency tree, and use the Inject decorator to provide an instance into a class constructor (this is covered later in this post).

Creating The Service

I recently worked on refactoring some code where in a component it had some code to handle getting a session object from sessionStorage. When working with Angular’s ng serve making changes will often result in a refresh, destroying any existing session. This feature allowed session storage to be turned on and keep it between refreshes by allowing the component to check if a value existed in sessionStorage object.

Using the principles in SOLID I moved the code into a service. Having a service do one thing it made the code easier to understand. The unit test for this is also smaller and can cover most of the component.

The service has one additional property which is storagePrefix, used to make each instance of the service unique to it’s component.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
import { Injectable } from '@angular/core';

@Injectable()
export class ComponentService {
  private storagePrefix = 'my-app';

  public init(storagePrefix: string) {
    this.storagePrefix = storagePrefix;
  }

  public load(key: string): string | undefined {
    return sessionStorage.getItem(`${this.storagePrefix}-${key}`);
  }

  public save(key: string, value: string): void {
    sessionStorage.setItem(`${this.storagePrefix}-${key}`, value);
  }
}

In it’s current implementation it’s usable, but it has a few issues. The main one is that it uses sessionStorage but trusts that it exists on the global object, this means we should always expect it to be available, and is a bad practice. It also limits this service to only use sessionStorage and no other key/value store. It would be better if we could provide it to our service instead.

We can also set the storagePrefix via our app configuration and means our component doesn’t need to understand this configuration (yet!) so it avoids us having to call the init method within a component’s ngOnInit.

By making these injectable, we will not only be able to provide our sessionStorage but we can support any module or library that conforms to the Web Storage API . This means we no longer need to rely on the global object and we have more control over what we inject.

The powerful feature of dependency injection is the ability to pass instances of other non-angular applications. For example you could pass the instance of a jQuery application and call it’s methods from your application (you’ll also want to check out NgZone when running these, but I’ll discuss that in a future post).

This allows for the transition of applications over time within an organisation, rather than having to replace all applications in one go. It’s a good way to safely inject features from third-party libraries and means passing mock versions is easier for tests.

Creating an injectable service and module

Making features injectable

First we need to define the interface of the object want to use for the configuration. This interface provides a contract in the forRoot method described below and tells developers what options they can provide to the application

Our configuration will require a storagePrefix as a string - this is need to set the name of the store. We also pass our Storage instance, but with this we can set a default value in our component

1
2
3
4
export interface ComponentServiceConfig {
  storagePrefix: string;
  storageInterface?: Storage;
}

Next we will create an InjectionToken, a token that is usually constant representation of something we want to provide via dependency injection. You can use any type of object as your token type, and can even provide a factory method to provide dependencies. Check out the Tree-shakeable Injection Token to see how you can do some advanced use. In our case we will just have the component config object.

1
2
import { InjectionToken } from '@angular/core';
const COMPONENT_SERVICE_CONFIG = new InjectionToken<ComponentServiceConfig>('COMPONENT_SERVICE_CONFIG');

Now we have the token service we will provide a simple configuration with this token which will be an object, but a token can be used to inject anything from a string to a function, or even a class instance.

Making the Angular module configurable

To make the service configurable, it should be provided via a NgModule - a regular module might look something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import { NgModule } from '@angular/core';
import { CommonModule } from '@angular/common';
import { ReactiveFormsModule } from '@angular/forms';
// Module specific imports ...

@NgModule({
  imports: [CommonModule, ReactiveFormsModule],
  declarations: [FeatureFormComponent, FeatureViewComponent, FeaturePipe],
  exports: [FeatureFormComponent, FeatureViewComponent],
  provides: [ComponentService]
})
export class LibraryModule {}

To make this module accept a configuration we need to add a method that can return an instance of our module with custom providers. Providers are an important part of Angular’s dependency injection as these are the instance of what child components can inject - all will hopefully become clear through the code example.

First we will add a static forRoot method to the class. This method provides a factory function where we can pass a configuration in the application imports.

The factory returns an instance of that module with the provided configuration, and this passed into the items with Inject . The module can then be further used without the forRoot as long as it’s configured at the root of the application (see SkipSelf )

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import { NgModule, ModuleWithProviders, Optional, SkipSelf } from '@angular/core';

// Here we can create a default object
const defaultConfig: ComponentServiceConfig = {
  storagePrefix: '',
  storageInterface: sessionStorage; // In our default value we will trust the global is there 
}

@NgModule({
  ...
})
export class LibraryModule {
  constructor(
    @Optional()
    @SkipSelf()
    parentModule: LibraryModule,
  ) {}
  static forRoot(config?: ComponentServiceConfig): ModuleWithProviders {
    return {
      ngModule: LibraryModule,
      providers: [
        {
          provide: COMPONENT_SERVICE_CONFIG,
          // Destructring allows the value of the config to be merged
          // this is a shallow operation so keep the objects simple
          useValue: { ...defaultConfig, ...config },
        },
      ],
    };
  }
}

Before we can use this in our application we also need to update the ComponentService itself to use this configuration. Here the Inject decorator will be used to tell the constructor that the ComponentServiceConfig this service needs will be available to inject via the COMPONENT_SERVICE_CONFIG token.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
import { Inject } from '@angular/core';

export class ComponentService {
  constructor(@Inject(COMPONENT_SERVICE_CONFIG) private readonly config: ComponentServiceConfig) {}

  public load(key: string): string | undefined {
    return this.config.storageInterface.getItem(`${this.config.storagePrefix}-${key}`);
  }

  public save(key: string, value: string): void {
    this.config.storageInterface.setItem(`${this.config.storagePrefix}-${key}`, value);
  }
}

As the config is set with private in the constructor, TypeScript knows to set this as a property on the class. This is the same as writing the following code and provides a clean shorthand to set up properties.

1
2
3
4
5
6
7
export class ComponentService {
  private config: ComponentServiceConfig;

  constructor(config: ComponentServiceConfig) {
    this.config = config;
  }
}

Providing the features in the app

Now that we have our configurable module, the Angular application can be provided with a configuration when importing.

As the module now accepts any Storage type, we’ll instead provide localStorage for this application, along side the storagePrefix:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
@NgModule({
  declarations: [AppComponent],
  imports: [
    BrowserModule,
    RouterModule.forRoot([], routerConfig),
    LibraryModule.forRoot({
      storageInterface: localStorage,
      storagePrefix: 'my-new-app'
    })
  ],
  providers: [...environment.mockProviders],
  bootstrap: [AppComponent]
})
export class AppModule {}

Another example is we could have an NPM library that provides the same interface, such as localStorage-ponyfill , it could be used in your application - or you may want to provide it to a TestBed configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// Using it in our application
import { createLocalStorage } from "localstorage-ponyfill";
const fakeLocalStorage = createLocalStorage({ mode : "memory" });

@NgModule({
  imports: [
    ...
    LibraryModule.forRoot({
      storageInterface: fakeLocalStorage,
      storagePrefix: 'my-new-app'
    })
    ...
  ]
})
export class AppModule {}

// Or writing some tests

TestBed.configureTestingModule({
  declarations: [TestHostComponent],
  providers: [
    {
      provide: BB_FORMS_CONTAINER_CONFIG,
      useValue: {
        storageStrategy: fakeLocalStorage,
      },
    },
  ]
});

With this technique, any kind of primitive or complex object, method or class can now be passed into your feature modules - and is a useful way to provide third-party libraries to your modules, configuring not just your services, but also components, pipes and stores.

Any class with a constructor (provided it’s in the same dependency tree) can use Inject with your exported token to access this configuration.

Hopefully you will also find this technique useful to improve the configurable and testable features of your module. Feel free to respond on twitter if you spot any typos or errors.