Kynetx Network Services (KNS) API

From KynetxDocs

Jump to: navigation, search

Initialization (`/js/`)

initialize (/js/shared/kobj-static.js) - Load shared libraries
initialize (/js/static/kobj-static.js) - Synonym for /js/shared/kobj-static.js. (decprecated)
initialize (/js/<ruleset-id>;<ruleset-id>;...<ruleset-id>/kobj.js) - Client specific library. Parameters:
- dataset - Client specific data to be loaded for this user.
- param - any other parameter. If more that one ruleset ID is given, parameters may be given a namespace equal to the RID. Namespaces and parameter names are separated by a colon.
dispatch (/js/dispatch/<ruleset-id>;<ruleset-id>;...<ruleset-id>/) - return list of sites that the subject rulesets act on.
datasets (/js/datasets/<ruleset-id>;<ruleset-id>;...<ruleset-id>/) - return a Javascript program that defines the datasets for this ruleset.
version (/js/version/<ruleset-id>) - Return the current build of service.
Parameters:
- flavor - What flavor of data to return. Valid values are html and json. Defaults to html.

Ruleset

The following methods are available for rulesets:

eval (/ruleset/eval/<ruleset-id>) - evaluate the ruleset with ID <ruleset-id>
Parameters:
- caller - the URL of the page that the ruleset is run against
- referer - the URL of the refering page to the caller
- kvars - any kvars set on the page
- title - the title of the caller
- mode - if set to 'test' then rules with rule state equal to test will be evaluated along with active rules.
flush (/ruleset/flush/<ruleset-id>) - flush the ruleset with ID <ruleset-id> from the cache.
console (/ruleset/console/<ruleset-id>) - display a debugging console for ruleset with ID <ruleset-id>.
describe (/ruleset/describe/<ruleset-id>) - display description and other meta data for ruleset with ID <ruleset-id>.
Parameters:
- flavor - What flavor of data to return. Valid values are html and json. Defaults to html.
version (/ruleset/version/<ruleset-id>) - display version information for service
Parameters:
- flavor - What flavor of data to return. Valid values are html and json. Defaults to html.

Not implemented yet:

/ruleset/get/<ruleset-id>
/ruleset/get/<ruleset-id>/rule/<rule-name>
/ruleset/put/<ruleset-id>

Callback

/callback/<ruleset-id> - log a callback event.
Parameters:
- url
- caller
version (/callback/version/) - display version information for service Parameters:
- <tt>flavor - What flavor of data to return. Valid values are html and json. Defaults to html.

Management

validate (/manage/validate/) - depricated in favor of parse method. validate and parse ruleset
Parameters:
- rule - Which ruleset to validate.
- flavor - What flavor of data to return. Valid values are html and json. Defaults to html.
jsontokrl (/manage/jsontokrl/) - return the KRL for a given JSON parsetree
Parameters:
- type - If set to bodyonly, expects the body of a rule, otherwise expects a full ruleset.
- flavor - What flavor of data to return. Valid values are html and json. Defaults to html.
version (/manage/version/<ruleset-id>) - display version information for service
Parameters:
- flavor - What flavor of data to return. Valid values are html and json. Defaults to html.

parse

KRL parser. This method has submethods as follows.
Each submethod takes identical parameters:

- krl - KRL to parse.
parse/ruleset (/manage/parse/ruleset) - KRL parser for a complete ruleset
parse/rule (/manage/parse/rule) - KRL parser for a single rule
parse/global (/manage/parse/global) - KRL parser for the global declarations
parse/dispatch (/manage/parse/dispatch) - KRL parser for the dispatch directive
parse/meta (/manage/parse/meta) - KRL parser for the meta directive

An error will be returned in a hash (record) with the key 'error'.

{"error" : "Something bad happened"}

You must use a POST to submit this.

unparse

KRL unparser. This method has submethods as follows.
Each submethod takes identical parameters:

- ast - Abstract syntax tree (expressed as JSON) to unparse.
unparse/ruleset (/manage/unparse/ruleset) - KRL unparser for a complete ruleset
unparse/rule (/manage/unparse/rule) - KRL unparser for a single rule
unparse/global (/manage/unparse/global) - KRL unparser for the global declarations
unparse/dispatch (/manage/unparse/dispatch) - KRL unparser for the dispatch directive
unparse/meta (/manage/unparse/meta) - KRL unparser for the meta directive

Analytics

TBD

SNS Administration

By subscribing different machines/KIBs/servers to the Kynetx SNS service (Topics), a single command may be issued to control a multiplicity of KNS/Apache aware services.

SNS Components

There are three components to SNS administration

Base perl module
SNS Endpoint to listen for notifications
Implementation utility

Perl Module

Module is located in Kynetx::Predicates::Amazon. SNS uses the same request and signature protocol as the Product Advertising API so I include Kynetx::Predicates::Amazon::RequestSignatureHelper.pm

I also created Kynetx::Predicates::Amazon::SNS::Response.pm to provide an object interface for the responses collected by the SNS endpoint.

SNS.pm

This is not quite the module that we talked about releasing to the Perl community. There are a number of Kynetx perl methods that I was unwilling to duplicate, so more work would be required to create a standalone SNS.pm, but nothing should be in SNS.pm that requires the KRL engine.

Methods

I support all the SNS actions except for AddPermission and RemovePermission. These are administration actions that could just as easily be accomplished through the Amazon CLI.

The implementation utility handles all of the parameter configuration. This section is just included for completeness.

usage

my $sns = Kynetx::Predicates::Amazon::SNS->new($param);

$sns->create_topic()  
$sns->delete_topic()  
$sns->get_topic_attributes()  
$sns->list_subscriptions()  
$sns->list_subscriptions_by_topic()  
$sns->list_topics()  
$sns->publish()  
$sns->set_topic_attributes($attr_hash)  
$sns->subscribe()  
$sns->unsubscribe()

$param is a hash of key-value pairs defined by the Amazon SNS API.

List of Amazon Keywords

Subject
TopicArn
Message
Action
Protocol
Endpoint
NewTopicName
AWSAccessKeyId
AWSSecretKey

SNS Endpoint

Some namespace collision here. SNS defines an endpoint as a target that is subscribed to a topic. Supported endpoint protocols are: http, https, email

The KNS management endpoint is an Apache/mod_perl handler (listener) that receives subscription confirmations and notifications from the Amazon SNS service.

The handler listens for get requests to http(s)://<host:port>/endpoint/

SNS notifications follow the form http(s)://<host:port>/endpoint/kns/sns

Further information is included in the request document in a directive format. Which is sufficient for admin purposes. For a proposed SNS/Event interface the url might look like http(s)://<host:port>/endpoint/sns_event/rid/version/ken

The request handler is Kynetx::Endpoints.pm

/kns/sns requests are passed to Kynetx::Endpoints::KNS.pm

KNS.pm listens for two types of SNS events: ConfirmSubscription and Notification.

SubscriptionConfirmation When you pass a subscription request to the SNS service, it sends a confirmation request to the the endpoint specified. Thus, it is critical to have the endpoint listener up and running if you are requesting a subscription to an http(s) protocol.

Upon receipt of a ConfirmSubscription request from Amazon, KNS.pm automatically replies (confirms) via the supplied link. Email endpoints will need to respond to the ConfirmSubscription request sent to the target email. Confirmations expire, so the response needs to be made within 24 hours of the message creation.

Notifications Notifications are generated by the Amazon SNS service in response to publish requests to a topic. Public requests must be signed by an AWS key. Publish permission can be limited to individual AWS ids via an Amazon SNS Policy. Policy implementation is left as an exercise for the SNS admin.

Notifications to the listener are signed (RSA) by Amazon and KNS.pl checks for a valid Amazon signature (SimpleNotificationService.pem) before parsing the notification.

Kynetx::Predicates::Amazon::SNS::Response.pm is responsible for parsing the GET request, extracting headers and content, verifying the signature and returning a simple SNS::Response.pm object.

Assuming that the Notification is valid, KNS.pm expects a directive document in the as the notification content. Here is the format of a flush request for rule a144x22

{
  'options' => {
    'action' => 'flush'
  },
  'name' => 'kns',
  'meta' => {
    'rid' => 'a144x22'
  }
}

Currently supported actions are:

flush
stats
count

KV pairs 'name' and 'action' are required parameters. 'name' must equal 'kns'. Notifications which don't meet these requirements will generate an "Invalid (SNS) directive" warning and will be ignored.

If the value of 'action' is not a recognized action, an "SNS Handler received unmatched notification:" error with the text of the notification appended is logged and the notification is ignored.

If no rid is specified, KNS will perform a flush_all command to memcached

SNS.pm Implementation Utility

Utility is called kns.pl. It is found in the /bin directory of the KRL Engine repository.

Usage

Manage KNS via SNS

Options:

    -h                  : display this message
    -l                  : List topics
    -a                  : List all subscriptions
    -s <topic>          : List subscriptions by Topic
    -v 0 | 1 | 2        : show detailed output (use -vv for highest level of detail)
    -m <count | stats>  : Send a 'cache status' message (requires: -T <topic>)

 *SNS management*

    -T <topic>                                  : Create SNS Topic
    -D <topic>                                  : Delete SNS Topic
    -S <IP address[:port] | email> -T <topic>   : Subscribe to Topic
    -P http | https | email                     : Protocol to use for subscription
    -N <displayname>                            : Change DisplayName (default: AWS Notifications)

I have tried to provide context sensitive error messages where possible, but I recommend that you don't do dumb things.

kns.pl supports the core SNS API management commands and KNS management directives. SNS commands are generally uppercase options, while KNS specific operations are lowercase.

When creating a new topic, you provide a plain text label and SNS creates a topic; ie: foo -> arn:aws:sns:us-east-1:791773988531:foo.

As a shortcut, when kns.pl requires a topic, I allow you to use kns.pl -T foo in addition to the cumbersome kns.pl -T arn:aws:sns:us-east-1:791773988531:foo

Honest to goodness examples

SNS Administration

Show help
    kns.pl -h

List Topics
    kns.pl -l

List all subscriptions
    kns.pl -a

List the subscriptions for a single topic (foo)
    kns.pl -s foo
    kns.pl -s arn:aws:sns:us-east-1:791773988531:foo

Enable debugging
    kns.pl -v1 ...

Enable even more debugging
    kns.pl -v2 ...

Disable debugging (this is the default, but an argument to -v is required and I chose '0' for symmetry)
    kns.pl -v0 ...

SNS management options

Create an SNS Topic named 'slartibartfast'
    kns.pl -T slartibartfast

Delete an SNS Topic
    kns.pl -D slartibartfast

Subscribe an email to a topic (slartibartfast)
    kns.pl -S autojam@kynetx.com -T slartibartfast

Subscribe a KRL listener to a topic
    kns.pl -S 64.55.47.131 -T slartibartfast

Subscribe a secure KRL listener to a topic
    kns.pl -S 64.55.47.131 -T slartibartfast -P https

Change the name that SNS uses for the 'from' field in email notifications
    kns.pl -T slartibartfast -N "Timmys Happy Time Grill"

KNS Administration

Send a cache flush request for rule a144x22
    kns.pl -m a144x22 -T flush

Send a cache flush all request
    kns.pl -m all -T flush

Send a request for servers to report their cache statistics
    kns.pl -m stats -T stats

Send a request for servers to report the number of items in their cache
    kns.pl -m count -T count

Kynetx Network Services (KNS) API

From KynetxDocs

Contents

Initialization (`/js/`)

Ruleset

Callback

Management

parse

unparse

Analytics

SNS Administration

SNS Components

Perl Module

SNS.pm

Methods

SNS Endpoint

SNS.pm Implementation Utility

Honest to goodness examples

Views

Personal tools

Navigation

Other

Search

Toolbox

Kynetx Network Services (KNS) API

From KynetxDocs

Contents

Initialization (/js/)

Ruleset

Callback

Management

parse

unparse

Analytics

SNS Administration

SNS Components

Perl Module

SNS.pm

Methods

SNS Endpoint

SNS.pm Implementation Utility

Honest to goodness examples

Views

Personal tools

Navigation

Other

Search

Toolbox

Initialization (`/js/`)