Choose XML, or a GUI, or CIBS?

Life in the heartbeat2 world seems to focus on either XML or a GUI. If you are a unix scripter and would like to operate easily on hertbeat2 resources I have developed an alternative: CIBS. CIBS stands for CIB Scripts, scripts that help you easily manipulate the content of the CIB. While these CIBS are by no means a replacement for all XML in heartbeat land or an understanding of how to use the heartbeat command line tools, they may make your life easier and allow you to easily edit resources without having to constantly edit XML.

A Resource CIBS is named after the type of resource or resource group that it will create and can be invoked with very simple options causing resources to be defined in the heartbeat CIB.

Example:

Add two new resources, example_A & example_B, which must be colocated, and example_B must start after example_A, and example_A should have a preference for machine foo:

 example + A +c example_B +l foo
 example + B +a example_A

This will generate the XML required to create both example_A and example_B along with the constraints specified and invoke cibadmin with the appropriate switches to process this XML and first clearing any matching entries from the CIB. Just to get a glimpse of how much easier this method is for simple stuff, here is the equivalent of the first command above (example + A +c example_B +l foo):

 echo '<rsc_colocation id="example_A_colocation_example_B" 
         from="example_A" to="example_B" score="INFINITY"/>' |\ 
 cibadmin -V -D -o constraints -p

 echo '<rsc_colocation id="example_A_colocation_example_B" 
         from="example_A" to="example_B" score="INFINITY"/>' |\ 
 cibadmin -V -C -o constraints -p

 echo '<rsc_location id="example_A_location" rsc="example_A">
        <rule id="example_A_location_machine" score="100">
          <expression id="example_A_location_machine_exp" 
            attribute="#uname" operation="eq" value="foo"/>
        </rule>
      </rsc_location>' |\ 
 cibadmin -V -D -o constraints -p

 echo '<rsc_location id="example_A_location" rsc="example_A">
        <rule id="example_A_location_machine" score="100">
          <expression id="example_A_location_machine_exp" 
            attribute="#uname" operation="eq" value="foo"/>
        </rule>
      </rsc_location>' |\ 
 cibadmin -V -C -o constraints -p

 echo '<primitive id="example_A" class="ocf" provider="heartbeat"
        type="Stateful">
        <instance_attributes id="example_A_ia">
          <attributes>
            <nvpair id="example_A_state" name="state" value="A"/>
          </attributes>
        </instance_attributes>
      </primitive>' |\ 
 cibadmin -V -D -o resources -p

 echo '<primitive id="example_A" class="ocf" 
         provider="heartbeat" type="Stateful">
        <instance_attributes id="example_A_ia">
          <attributes>
            <nvpair id="example_A_state" name="state" value="A"/>
          </attributes>
        </instance_attributes>
      </primitive>' |\ 
 cibadmin -V -C -o resources -p

If you later decide that you would be prefer example_A to start after example_B and you want the preferred machine to instead be bar, you can easily change this by issuing the following commands:

 example = B -a example_A
 example = A +a example_A -l foo +l bar

CIBS Usage & Arguments

The general usage of a resource CIBS is: options, followed by the resource shortname, followed by a list of constraints:

 resource_cibs [-nvu|--help] <+-=.>[m] shortname [<+-=.>C constraint ...]

  m    manual (experimental), call an OCF agent directly instead
       of cibadmin (only usefull for resoures, constraints are
       ignored)

The <+-=.> indicates what to do to the resource or constraint:

  +    add                                (if manual: OCF start)
  -    delete                             (if manual: OCF stop)
  =    maintain (no change)               (if manual: OCF status)
  .    output the related XML to stdout   (if manual: OCF monitor)

Note that you must always specify the shortname of the resource, use the = switch when you want to modify an already existing resource's constraints.

The current constraints are specified by: C, followed by the constraint argument, the C and its arguments can be:

 c colocated_to   fullname of resource to be colocated to
 b start_before   fullname of resource to start before
 a start_after    fullname of resource to start after
 l host           hostname of the preferred host to run on

It is important that fullname be the fullname of the another resource. Resource CIBS will generally generate a resource name that is longer than the shortname used above. This requires some knowledge of the of the XML output of your resource CIBS to know the fullname that they will generate. Generally top level (outer) resource fullnames will be of the format: <type>_<shortname>.

As a shortcut, arguments which are identical can be held until that last point. I.E. colocation and starting after another resource can be specified like this: +c +a resource

Options:

 -n       do not execute any actual cibadmin commands 
 -v       verbose, output each cibadmin invocation with its
          switches and the generated XML
 -u       print the usage
 --help   print the help message

Some of my resource CIBS

• drbd, used to setup a drbd resource so that it can be mounted by another resource
• vserver, creates a resource group that sets up a drbd device, mounts a vserver filesystem from that drbd device, and starts a vserver.
• drbddisk, used to setup a drbd device and mount the filesystem on it.

Note: these work best in combination with my OCF Agents

Resource CIBS Structure

Resource CIBS need to be written in a bash compatible language and contain at least one main function called, strangely enough: resource. This function takes two arguments, a long RESOURCE name and a SHORT resource name. With these two arguments this function needs to be able to generate (and output to stdout) the required XML to create your custom resource or resource group. The last thing that this CIBS needs to do is to "source" the res.lib library which will take care of command line processing and filling in the rest of the functionality required to make a complete resource CIBS.

res.lib

The primary operating script on this page is not actually a shell script, but a shell script library. This library is meant to be "sourced" by a very simple custom shell script, called a resource script, for each type of resource or resource group that you would like to control with heartbeat

Example Resource CIBS

This CIBS uses the example OCF agent Statefull provided by heartbeat. With this exmaple you should be able to easily create scenarios and test out constraints without using real resources yet.

 #!/bin/sh

 resource()
 { # RES short
 cat - <<EOF
      <primitive id="${1}" class="ocf" provider="heartbeat" type="Stateful">
        <instance_attributes id="${1}_ia">
          <attributes>
            <nvpair id="${1}_state" name="state" value="$2"/>
          </attributes>
        </instance_attributes>
      </primitive>
 EOF
 }

 . res.lib

Future

While these CIBS are rather simplistic (that is the objective), they may be too limited for your needs. For example, there are only some very basic constraint types available. It is easy to write new resource CIBS to deal with new types of resources, but there should probably be an easy method to also integrate new constraints types, a simple plugin API. Also there are some standard simple constraint types that should probably added to the current constraints types.

Objective

1. The primary objective of these CIBS is the simplicity of the command line invocation.

Any new development must keep that objective in mind. While I do believe there is lots of room for additions to these since people may use many different resource types, and have some of their own particular common constraint types, any improvements will need to consider objective #1.

Fullnames

Are fullnames really just a type and a name?

Extra arguments

Perhaps they should be comma separated?

New Constraints

  -t conflict        fullname of resource which resource cannot be colocated with
  -v valid_hosts...   list of hosts on which resource must run on
  -i invalid_hosts... list of hosts on which resource must not run on

Although, the last two really really just an "ORed" list of location constraints with +/-INFINITY values. If we could use variable arguments, we could just add the list to those constraints and set the +/- as a value to each one. Even so, I would think that absolutes (+/-INFINITY) probably merit their own shortcut constraints since we probably do not have that many constraint types, do we?

Constraint Types Plugin API

To not get too complicated, a constraint type should have a way of registering its function, its arguments (allow more than one) and its invocation switch (what goes after the <+-=.>). This would allow resource CIBS to register their own constraint types.

Another feature might include the ability to access explicit constraint types from the command line. Something like:

  <+-=.>[#constraint] [arguments]

The # constraint would require a special markup in a file to parse out the constraint, potentially surrounding it by special comments.

An alternative would be a simple sourcing ability which would source a file from the command line:

  --source path +C sourced_constraint_arg