Discovering and Using OPC UA Thermometers
OPC UA is a communication protocol for industrial automation. It is a client/server technology that comes with a security and communication framework. This demo will help you get started using Akri to discover OPC UA PLC Servers and utilize them via a broker that contains an OPC UA Client. Specifically, a Akri Configuration called OPC UA Monitoring was created for this scenario, which will show how Akri can be used to detect anomaly values of a specific OPC UA Variable. To do so, the OPC UA Clients in the brokers will subscribe to that variable and serve its value over gRPC for an anomaly detection web application to consume. This Configuration could be used to monitor a barometer, CO detector, and more; however, for this example, that variable will represent the PLC values for temperature of a thermostat and any value outside the range of 70-80 degrees is an anomaly.
The demo consists of the following components:
Two OPC UA PLC Servers
(Optional) Certificates for the Servers and Akri brokers
An OPC UA Monitoring broker that contains an OPC UA Client that subscribes to a specific NodeID (for that PLC variable)
Akri installation
An anomaly detection web application
Demo Flow
An operator (meaning you!) applies to a single-node cluster the OPC UA Configuration, which specifies the addresses of the OPC UA Servers, which OPC UA Variable to monitor, and whether to use security.
Agent sees the OPC UA Configuration, discovers the servers specified in the Configuration, and creates an Instance for each server.
The Akri Controller sees the Instances in etcd and schedules an OPC UA Monitoring broker pod for each server.
Once the OPC UA Monitoring broker pod starts up, it will create an OPC UA Client that will create a secure channel with its server.
The OPC UA Client will subscribe to the OPC UA Variable with the NodeID with
Identifier
"FastUInt1" andNamespaceIndex
2 as specified in the OPC UA Configuration. The server will publish any time the value of that variable changes.The OPC UA Monitoring broker will serve over gRPC the latest value of the OPC UA Variable and the address of the OPC UA Server that published the value.
The anomaly detection web application will test whether that value is an outlier to its pre-configured dataset. It then will display a log of the values on a web application, showing outliers in red and normal values in green.
The following steps need to be completed to run the demo:
If at any point in the demo, you want to dive deeper into OPC UA or clarify a term, you can reference the online OPC UA specifications.
Setting up a cluster
Reference our cluster setup documentation to set up a cluster for this demo. For ease of setup, only create a single-node cluster, so if installing K3s or MicroK8s, you can skip the last step of the installation instructions of adding additional nodes. If you have an existing cluster, feel free to leverage it for the demo. This documentation assumes you are using a single-node cluster; however, you can certainly use a multi-node cluster.
Creating X.509 v3 Certificates
If security is not desired, skip to Creating OPC UA Servers, as each monitoring broker will use an OPC UA Security Policy of None if it cannot find credentials mounted in its pod.
Akri will deploy an OPC UA Monitoring broker for each OPC UA Server a node in the cluster can see. This broker contains an OPC UA Client that will need the proper credentials in order to communicate with the OPC UA Server in a secure fashion. Specifically, before establishing a session, an OPC UA Client and Server must create a secure channel over the communication layer to ensure message integrity, confidentiality, and application authentication. Proper application credentials in the form of X.509 v3 certificates are needed for application authentication.
Every OPC UA Application, whether Client, Server, or DiscoveryServer, has a certificate store, which includes the application's own credentials along with a list of trusted and rejected application instance certificates. According to OPC UA specification, there are three ways to configure OPC UA Server and Clients' certificate stores so that they trust each other's certificates, which are explained in the OPC UA proposal. This demo will walk through the third method of creating Client and Server certificates that are issued by a common Certificate Authority (CA). Then, that root CA certificate simply needs to be added to the trusted folder of Client and Servers' certificate stores, and they will automatically trust each other on the basis of having a common root certificate. The following image walks through how to configure the Client and Server certificate stores for Akri.
Generate an X.509 v3 Certificate for Akri OPC UA Monitoring brokers and sign it with the same CA that has signed the certificates of all the OPC UA Servers that will be discovered.
Create a Kubernetes Secret named opcua-broker-credentials that contains four items with the following key names: client_certificate, client_key, ca_certificate, and ca_crl.
The credentials will be mounted in the broker at the path /etc/opcua-certs/client-pki.
Generating certificates
Create three (one for the broker and each server) OPC UA compliant X.509v3 certificates, ensuring that the certificate contains the necessary components such as an application URI. They should all be signed by a common Certificate Authority (CA). There are many tools for generating proper certificates for OPC UA, such as the OPC Foundation's Certificate Generator or openssl (as in this walk through).
Creating an opcua-broker-credentials Kubernetes Secret
The OPC UA Client certificate will be passed to the OPC UA Monitoring broker as a Kubernetes Secret mounted as a volume. Read more about the decision to use Kubernetes secrets to pass the Client certificates in the Credentials Passing Proposal. Create a Kubernetes Secret, projecting each certificate/crl/private key with the expected key name (i.e. client_certificate
, client_key
, ca_certificate
, and ca_crl
). Specify the file paths such that they point to the credentials made in the previous section.
When mounting certificates is enabled later in the Running Akri section with Helm via --set opcua.configuration.mountCertificates='true'
, the secret named opcua-broker-credentials
will be mounted into the OPC UA monitoring brokers. It is mounted to the volume credentials
at the mountPath
/etc/opcua-certs/client-pki, as shown in the OPC UA Configuration Helm template. This is the path where the brokers expect to find the certificates.
Creating OPC UA Servers
Now, we must create some OPC UA PLC Servers to discover. Instead of starting from scratch, we deploy OPC PLC server containers. You can read more about the containers and their parameters here.
Create an empty YAML file called
opc-deployment.yaml
.(Optional) If you are using security, place the OpcPlc certificate and the CA certificate as below.
(A) If you are not using security, copy and paste the contents below into the YAML file.
(B) If you are using security, copy and paste the contents below into the YAML file, replacing the path in the last line with your path to the folder that contains the certificates.
Save the file, then simply apply your deployment YAML to create two OPC UA servers.
We have successfully created two OPC UA PLC servers, each with one fast PLC node which generates an unsigned integer with lower bound = 65 and upper bound = 85 at a rate of 1. It should be up and running.
Running Akri
Make sure your OPC UA PLC Servers are running.
Now it is time to install the Akri using Helm. When installing Akri, we can specify that we want to deploy the OPC UA Discovery Handlers by setting the helm value
opcua.discovery.enabled=true
. We also specify that we want to create an OPC UA Configuration with--set opcua.configuration.enabled=true
. In the Configuration, any values that should be set as environment variables in brokers can be set inopcua.configuration.brokerProperties
. In this scenario, we will specify theIdentifier
andNamespaceIndex
of the NodeID we want the brokers to monitor. In our case that is our temperature variable we made earlier, which has anIdentifier
ofFastUInt1
andNamespaceIndex
of2
. Your OPC PLC discovery URL will look something like"opc.tcp://<host IP address>:50000/
. If using security, uncomment--set opcua.configuration.mountCertificates='true'
.Note: See the cluster setup steps for information on how to set the crictl configuration variable
AKRI_HELM_CRICTL_CONFIGURATION
Note:
FastUInt1
is the identifier of the fast changing node that is provided by the OPC PLC server.Akri Agent will discover the two Servers and create an Instance for each Server. Watch two broker pods spin up, one for each Server.
To inspect more of the elements of Akri:
Run
kubectl get crd
, and you should see the CRDs listed.Run
kubectl get akric
, and you should seeakri-opcua-monitoring
.If the OPC PLC Servers were discovered and pods spun up, the instances can be seen by running
kubectl get akrii
and further inspected by runningkubectl get akrii akri-opcua-monitoring-<ID> -o yaml
Deploying an anomaly detection web application as an end consumer of the brokers
A sample anomaly detection web application was created for this end-to-end demo. It has a gRPC stub that calls the brokers' gRPC services, getting the latest temperature value. It then determines whether this value is an outlier to the dataset using the Local Outlier Factor strategy. The dataset is simply a csv with the numbers between 70-80 repeated several times; therefore, any value significantly outside this range will be seen as an outlier. The web application serves as a log, displaying all the temperature values and the address of the OPC UA Server that sent the values. It shows anomaly values in red. The anomalies always have a value of 120 due to how we set up the DoSimulation
function in the OPC UA Servers.
Deploy the anomaly detection app and watch a pod spin up for the app.
Determine which port the service is running on. Be sure to save this port number for the next step.
SSH port forwarding can be used to access the streaming application. In a new terminal, enter your ssh command to to access your VM followed by the port forwarding request. The following command will use port 50000 on the host. Feel free to change it if it is not available. Be sure to replace
<anomaly-app-port>
with the port number outputted in the previous step.Note we've noticed issues with port forwarding with WSL 2. Please use a different terminal.
Navigate to
http://localhost:50000/
. It takes 3 seconds for the site to load, after which, you should see a log of the temperature values, which updates every few seconds. Note how the values are coming from two different DiscoveryURLs, namely the ones for each of the two OPC UA Servers.
Clean up
Delete the anomaly detection application deployment and service.
Delete the OPC UA Monitoring Configuration and watch the instances, pods, and services be deleted.
Bring down the Akri Agent, Controller, and CRDs.
Delete the OPC UA server deployment.
Extensions
Now that you have the end to end demo running let's talk about some ways you can go beyond the demo to better understand the advantages of Akri. This section will cover:
Adding a node to the cluster
Using a Local Discovery Server to discover the Servers instead of passing the DiscoveryURLs to the OPC UA Monitoring Configuration
Modifying the OPC UA Configuration to filter out an OPC UA Server
Creating a different broker and end application
Creating a new OPC UA Configuration
Adding a Node to the cluster
To see how Akri easily scales as nodes are added to the cluster, add another node to your (K3s, MicroK8s, or vanilla Kubernetes) cluster. 1. If you are using MicroK8s, create another MicroK8s instance, following the same steps as in Setting up a single-node cluster above. Then, in your first VM that is currently running Akri, get the join command by running microk8s add-node
. In your new VM, run one of the join commands outputted in the previous step.
Confirm that you have successfully added a node to the cluster by running the following in your control plane VM:
You can see that another Agent pod has been deployed to the new node; however, no new OPC UA Monitoring brokers have been deployed. This is because the default
capacity
for OPC UA is 1, so by default only one Node is allowed to utilize a device via a broker.Let's play around with the capacity value and use the
helm upgrade
command to modify our OPC UA Monitoring Configuration such that the capacity is 2. On the control plane node, run the following, once again uncommenting--set opcua.configuration.mountCertificates='true'
if using security. Watch as the broker terminates and then four come online in a Running state.Once you are done using Akri, you can remove your worker node from the cluster. For MicroK8s this is done by running on the worker node:
Then, to complete the node removal, on the host run the following, inserting the name of the worker node (you can look it up with
microk8s kubectl get no
):
Setting up and using a Local Discovery Server (Windows Only)
This walk-through only supports setting up an LDS on Windows, since that is the OS the OPC Foundation sample LDS executable was written for.
A Local Discovery Server (LDS) is a unique type of OPC UA server which maintains a list of OPC UA servers that have registered with it. The OPC UA Configuration takes in a list of DiscoveryURLs, whether for LDSes or a specific servers. Rather than having to pass in the DiscoveryURL for every OPC UA Server you want Akri to discover and deploy brokers to, you can set up a Local Discovery Server on the machine your servers are running on, make the servers register with the LDS on start up, and pass only the LDS DiscoveryURL into the OPC UA Monitoring Configuration. Agent will ask the LDS for the addresses of all the servers registered with it and the demo continues as it would've without an LDS.
The OPC Foundation has provided a Windows based LDS executable which can be downloaded from their website. Download version 1.03.401. It runs as a background service on Windows and can be started or stopped under Windows -> Services. The OPC Foundation has provided documentation on configuring your LDS. Most importantly, it states that you must add the LDS executable to your firewall as an inbound rule.
Make sure you have restarted your OPC UA Servers, since they attempt to register with their LDS on start up. Now, we can install Akri with the OPC UA Configuration, passing in the LDS DiscoveryURL instead of both servers' DiscoveryURLs. Replace "Windows host IP address" with the IP address of the Windows machine you installed the LDS on (and is hosting the servers). Be sure to uncomment mounting certificates if you are enabling security:
You can watch as an Instance is created for each Server and two broker pods are spun up.
Modifying the OPC UA Configuration to filter out an OPC UA Server
Instead of deploying brokers to all servers registered with specified Local Discovery Servers, an operator can choose to include or exclude a list of application names (the applicationName
property of a server's ApplicationDescription
as specified by UA Specification 12). For example, to discover all servers registered with the default LDS except for the server named "SomeServer0", do the following.
Note: See the cluster setup steps for information on how to set the crictl configuration variable
AKRI_HELM_CRICTL_CONFIGURATION
Alternatively, to only discover the server named "SomeServer0", do the following:
Creating a different broker and end application
The OPC UA Monitoring broker and anomaly detection application support a very specific scenario: monitoring an OPC UA Variable for anomalies. The workload or broker you want to deploy to discovered OPC UA Servers may be different. OPC UA Servers' address spaces are widely varied, so the options for broker implementations are endless. Passing the NodeID Identifier
and NamespaceIndex
as environment variables may still suit your needs; however, if targeting one NodeID is too limiting or irrelevant, instead of passing a specific NodeID to your broker Pods, you could specify any other environment variables via --set opcua.configuration.brokerProperties.KEY='VALUE'
. Or, your broker may not need additional information passed to it at all. Decide whether to pass environment variables, what servers to discover, and set the broker pod image to be your container image, say ghcr.io/<USERNAME>/opcua-broker
.
Note: set
opcua.configuration.brokerPod.image.tag
to specify an image tag (defaults tolatest
).
Now, your broker will be deployed to all discovered OPC UA servers. Next, you can create a Kubernetes deployment for your own end application like anomaly-detection-app.yaml and apply it to your Kubernetes cluster.
Creating a new OPC UA Configuration
Helm allows us to parametrize the commonly modified fields in our Configuration files, and we have provided many. Run helm inspect values akri-helm-charts/akri
to see what values of the generic OPC UA Configuration can be customized, such as the Configuration and Instance ServiceSpec
s, capacity
, and broker PodSpec
. We saw in the previous section how broker Pod environment variables can be specified via --set opcua.configuration.brokerProperties.KEY='VALUE'
. For more advanced configuration changes that are not aided by the generic OPC UA Configuration Helm chart, such as credentials naming, we suggest downloading the OPC UA Configuration file using Helm and then manually modifying it. See the documentation on customizing an Akri installation for more details.
Last updated