Configuration
The configuration file used by ops
(such as one passed as a parameter, e.g. ops --config myconfig.json
) specifies various options and attributes of code execution, such as files to include, arguments, and environment variables. This file follows the standard json format.
For a complete sample of a configuration, see our sample.
You can also enable config interpolation for things like secrets, hosts or other configuration that you would like to inject from your environment. This is helpful to reduce duplicate config:
For example:
Will populate 'secret' with 'test':
Configuration Attributes
OPS_HOME
The OPS_HOME environment variable points to the location of where ops will store build images and releases locally. By default it is set to ~/.ops . However there are some use-cases where you might want it set to something else like so:
Args
Args
is an array of arguments passed to your program to execute when running the image. Most programs will consider the program name as arg[0] but not all.
DisableArgsCopy
DisableArgsCopy
, when set to true, disables the auto copy of files from host to image when present in args.
BaseVolumeSz
The BaseVolumeSz
is an optional parameter you can pass to specify the size of the base volume. By default the size is the end of blocks written by TFS.
To specify 100 megabytes:
To specify 1 gigabyte:
Boot
Boot
sets the path of bootloader file.
Uefi
Uefi
indicates whether image should support booting via UEFI. Default is false
.
UefiBoot
UefiBoot
sets the path of UEFI bootloader file.
BuildDir
BuildDir
path to the temporary directory used during image build.
CloudConfig
The CloudConfig
configures various attributes about the cloud provider, we want to use with ops.
BucketName
BucketName
specifies the bucket to store the Ops built image artifacts.
BucketNamespace
BucketNamespace
is required on uploading files to cloud providers as oci.
DedicatedHostId
This option is supported on AWS only. Please keep in mind that, depending on the dedicated host type, only some instance flavors may be available.
DedicatedHostId
is used to specify the dedicated host id placement on cloud provider.
DomainName
This option is supported on AWS, GCP and Vultr.
DomainName
is used to update a A record DNS entry with the started instance IP.
StaticIP
This option is only supported on AWS, GCP.
StaticIP
by cloud provider to assign a public static IP to a NIC.
EnableIPv6
This option is only supported on AWS, GCP.
If EnableIPv6
is set to true and a VPC is created, the new VPC will have ipv6 support. Otherwise, it doesn't affect the selected VPC ipv6 support.
Flavor
Flavor
specifies the machine type used to create an instance. Each cloud provider has different types descriptions.
ImageType
ImageType
allows the user to specify an image type (whose possible values are target platform-specific) when creating an image. Can be used to create images for Hyper-V generation 2 instances.
ImageName
ImageName
specifies the image name in the cloud provider.
It might be used by ops to set "IMAGE_NAME"
and "RADAR_IMAGE_NAME"
environment variables.
InstanceProfile
InstanceProfile
sets up an IAM role for an instance. Currently, this is only used for AWS but, in the future, it might be used to set roles for other cloud providers.
KMS
KMS
optionally encrypts AMIs if set. 'default' may be used for the default key or a KMS arn may be specified. This is only used for AWS.
Platform
Platform
defines the cloud provider to use with the ops CLI.
Currently supported platforms:
Default
onprem
Google Cloud Platform
gcp
AWS
aws
Vultr
vultr
Vmware
vsphere
Azure
azure
Openstack
openstack
Upcloud
upcloud
Hyper-v
hyper-v
DigitalOcean
do
OpenShift
openshift
Oracle Cloud Infrastructure
oci
Oracle VM VirtualBox
vbox
Proxmox VE
proxmox
See further instructions about the cloud provider in dedicated documentation page.
ProjectID
ProjectID
is used in some cloud providers to identify a workspace.
RootVolume
RootVolume
is an optional set of options one can apply to the root/base volume of an instance on AWS. These settings include the name, typeof, iops, size, and throughput.
For more information see the AWS page.
SecurityGroup
SecurityGroup
allows an instance to use an existing security group in the cloud provider.
On AWS, both the name and the id of the security group may be used as value.
Subnet
Subnet
allows an instance to use an existing subnet in the cloud provider.
On AWS, both the name and the id of the subnet may be used as value.
Tags
Tags
is a list of keys and values to provide more context about an instance or an image. There are a set of pre-defined tags to identify the resources created by ops
.
The following example demonstrates the default format used when deploying on different cloud providers.
On GCP (only) there is also an extended configuration attribute
that can be used to detail/control the purpose and destination of the tag
VPC
VPC
allows instance to use an existing vpc in the cloud provider.
On AWS and DO, both the name and the id of the vpc may be used as value. On DO setting the id (uuid) avoids an extra call to the api at the cost of readability.
Zone
Zone
is used in some cloud providers to identify the location where cloud resources are stored.
Dirs
Dirs
defines an array of directory locations to include into the image.
File layout on local host machine:
File layout on VM:
Env
Env
defines a map of environment variables to specify for the image runtime.
Files
Files
defines an array of file locations to include into the image.
Kernel
Kernel
sets the path of kernel image file.
KlibDir
KlibDir
sets the host directory where kernel libs are located.
Klibs
Defines a list of klibs to include. For example to run the NTP klib (eg: ntpd):
MapDirs
MapDirs
sets map of a local directory to a different path on the guest VM. For example the below adds all files under /etc/ssl/certs
on host to /usr/lib/ssl/certs
on VM.
Mounts
Mounts
is used to mount a volume in an instance.
In the case of onprem
provider, the volume can be a host directory (as opposed to a pre-made volume) that the guest will mount as a 9P filesystem.
See further instructions about volumes in dedicated documentation page.
NameServers
NameServers
is an array of DNS servers to use for DNS resolution. By default it is Google's 8.8.8.8
.
NanosVersion
NanosVersion
sets nanos version to be used on image manifest.
NightlyBuild
NightlyBuild
flag forces the use of latest dev builds.
NoTrace
NoTrace
is an array of syscalls to mute tracing for.
Program
Program
specifies the path of the program to refer to on attach/detach.
ProgramPath
ProgramPath
specifies the original path of the program to refer to on attach/detach.
RebootOnExit
RebootOnExit
reboot your application immediately if it crashes (exit code is not 0). Is turned off by default, but you can enable it.
same as
LocalFilesParentDirectory
LocalFilesParentDirectory
is the parent directory of the files/directories specified in Files and Dirs. The default value is the directory from where the ops command is running
TargetRoot
TargetRoot
TODO
VolumesDir
VolumesDir
is the directory used to store and fetch volumes.
PackageBaseURL
PackageBaseURL
provides the URL for downloading the packages.
PackageManifestURL
PackageManifestURL
provides the URL to download the manifest file that stores info about all packages.
Version
Version
ops package version. Like docker if the user doesn't provide version of the image we consider "latest" as the version.
Language
Language
ops package language.
Runtime
Runtime
ops package runtime.
Description
Description
ops package description.
TargetConfig
TargetConfig
provides a limited support to adding cloud provider specific config. This is done as to not overload the other configuration settings.
Currently only the proxmox
target has support here. For example:
RunConfig
The RunConfig
configures various attributes about the runtime of the ops instance, such as allocated memory and exposed ports.
Accel
Defines whether hardware acceleration should be enabled in qemu. This option is enabled by default, but will be disabled when Debug is set to true.
AtExit
Defines hooks to be ran after instance stops.
Bridged
Connects the unikernel network interface to a bridge with the name br0
. The bridge name may be overriden with the property BridgeName.
BridgeName
Connects the unikernel network interface to a bridge with the name specified. If the bridge does not exist in host machine it is created.
CanIPForward
Enable IP forwarding when creating an instance. Only GCP, Azure provider.
CPUs
Specifies the number of CPU cores the unikernel is allowed to use.
GPUs
Specifies the number of GPUs the unikernel is allowed to use. Only GCP provider.
GPUType
Specifies the type of GPU available. Only GCP provider.
Debug
Opens a port in unikernel to allow a connection with the GDB debugger. See further instructions in Debugging. If debug is set to true the hardware acceleration Accel is disabled.
Gateway
Defines the default gateway IP of the network interface.
GdbPort
Define the gdb debugger port. It only takes effect if debug is enabled. By default the GdbPort
is 1234
.
Imagename
Sets the name of the image file.
InstanceName
Sets the name of the instance.
InstanceGroup
Sets the group of the instance.
IPAddress
Defines the IP address of the network interface.
The IP address has to be specified along with the netmask and the gateway, so the unikernel can assign the IP address to the network interface.
IPv6Address
Defines the static IPv6 address of the network interface.
Memory
Configures the amount of memory to allocated to qemu
. Default is 128 MiB. Optionally, a suffix of "M" or "G" can be used to signify a value in megabytes or gigabytes respectively.
Vga
Defines whether to emulate a VGA output device in qemu
.
Mounts
Defines a list of directories paths in the host machine whose data will be copied to the unikernel.
NetMask
Defines the netmask of the network interface.
Nics
Is a list of pre-configured network cards.Meant to eventually deprecate the existing single-nic configuration. Currently only supported for Proxmox
Background
Starts unikernels in background. You can stop the unikernel using the onprem instances stop command.
Ports
A list of ports to expose. Alternatively, you can also use -p
in the command line.
ShowDebug
Enables printing more details about what ops is doing at the moment. Also, enables the printing of warnings and errors.
ShowErrors
Enables printing errors with more details.
ShowWarnings
Enables printing warnings details.
TapName
Connects the unikernel to a network interface with the name specified. If the tap
does not exist in host machine it is created.
UDPPorts
Opens ports that use UDP protocol.
Verbose
Enables verbose logging for the runtime environment. As of now, it prints the command used to start qemu
.
VolumeSizeInGb
This property is only used by cloud provider openstack and sets the instance volume size. Default size is 1 GB.
AttachVolumeOnInstanceCreate
This property is only used by cloud provider gcp to attach an existing cloud disk/volume to the instance being created.
ManifestPassthrough
There is the concept of the manifest in Nanos where there exists many other config options that don't boil up to ops configuration, however, sometimes you still wish to pass these settings down to the manifest. Certain klibs, in particular, have variables that need to be set. To set them do this:
Consoles
Nanos offers a 'consoles' feature that allows multiple methods for monitoring a running unikernel. By default, the 'serial' and 'vga' consoles are enabled for testing purposes. However, in a production environment, it is highly advised against using these consoles as they significantly slow down the unikernel, rendering it nearly non-functional.
To activate or deactivate consoles, you can use the '+' or '-' symbols. Here is an example configuration:
In the above example, +net
enables the net console locally for debugging purposes. The net console is faster than 'serial' when it comes to testing performance.
The net console also provides two options: netconsole_port
and netconsole_ip
. These options allow you to send logs to a remote machine for debugging. However, it is advisable to use 'syslog' instead of net console for production workloads.
Exec Protection
Nanos has an 'exec protection' feature that prevents the kernel from executing any code outside the main executable and other 'trusted' files explicitly marked. The program is further limited from modifying the executable file and creating new ones. This flag may also be used on individual files within the children tuple. This prevents the application from exec-mapping anything that is not explicitly mapped as executable.
This is not on by default, however, as many JITs won't work with it turned on.
CWD
Some applications expect to have a working directory in a different place than where they have been placed. You can adjust this via the manifest variable 'cwd':
Exec Wait For IPv4
This is an optional configuration setting that allows Nanos to wait for a valid ipv4 address to become available via DHCP using the timeout of 'exec_wait_for_ip4_secs'. If static ip is set than there is no effect. This configuration is not on by default.
This configuration can also be associated to a specific network interface, i.e:
Exec Wait For IPv6
This is an optional configuration setting that allows Nanos to wait for a valid ipv6 address to become available via DHCP using the timeout of 'exec_wait_for_ip6_secs'. If static ip is set than there is no effect. This configuration is not on by default.
This configuration can also be associated to a specific network interface, i.e:
expected_exit_code
This is an optional configuration setting that changes the program exit code to 0, if a match is found on the configuration.
Some possible configurations:
Mmap Min Address
This is an optional configuration setting that defines the minimum virtual address that a process is allowed to mmap. If set to zero, allow zero-page mappings to occur.
Program Overwrite Protection
By default, the user application is not allowed to overwrite the program binary file (and interpreter, if present). This default behavior can be overridden by inserting a program_overwrite
attribute in the root tuple of the manifest.
ltrace
This enables tracing calls made by the application binary to dynamic library functions. It works on both pie and no-pie programs, and also works with aslr.
mtu
This is an optional configuration setting that changes the MTU (Maximum Transmission Unit) size of the network interface(s). This doesn't affect TUN interface(s).
so_rcvbuf
This is an optional configuration setting used to manage the size (in bytes) of the socket receive buffer. The default buffer size is 208 KB, so to change the size to, say, 512 KB, you could use the following config:
static_map_program
This is an optional configuration setting that will disable demand paging. This is overriden when ltrace is in use.
trace
This is an optional configuration setting that allows specifying trace flags (i.e. a comma-delimited set of trace message types) in the trace
symbol of the root tuple. A given trace message is output only if its message type is enabled in the trace flags.
pf
- page-fault-related messages - setsTRACE_PAGE_FAULT
threadrun
- messages output when returning to user threads - setsTRACE_THREAD_RUN
all
- all tracing messages are enabledfor backward compatibility, any unknown trace flag enables all messages output via calls to thread_log(). - sets
TRACE_OTHER
Example: a "trace:pf,other"
value in the manifest enables page-fault-related messages and messages classified as "other".
reboot_on_exit
This is an optional configuration setting that reboots your application immediately if it stops with an exit code that matches the configuration.
This is checked after manifestpassthrough.expected_exit_code
execution (when both configured).
Note: poweroff/stop
operations work normally, regardless of the presence and value of the reboot_on_exit
option, and regardless of the application exit code.
Some possible configurations:
idle_on_exit
This is an optional configuration setting that keeps a VM running after the user program exits with an exit code that matches the configuration. The VCPUs remain halted.
Some possible configurations:
io-queues
By default the virtio-net driver utilizes as many tx/rx queues as supported by the attached device but limited to the number of cpus exposed to the instance. This setting may be changed via the following config for the guest:
uname
This is an optional configuration that will make the kernel return your chosen strings in the sysname and release fields of the uname syscall. Several programs expect known names or versions, to for example, enable certain features.
The default is set to "Nanos" for the sysname field, and "5.0-<NANOS_VERSION>" for the release.
transparent_hugepage
Transparent HugePage Support (THP) is enabled by default on nanos.
To disable THP support set "transparent_hugepage"
to "never"
. Any other value, will keep THP enabled.
Debugflags
Debugflags
adds additional debug flags to the runtime.
idle_on_exit
idle_on_exit
keeps a VM running after the user program exits successfully (exit code is 0). The VCPUs remain halted.
reboot_on_exit
reboot_on_exit
reboot your application immediately if it crashes (exit code is not 0). Is turned off by default, but you can enable it.
Force
TODO
Last updated