WSS - WebService Specification Schema and LL libraries

Yusuke Kawasaki【川﨑有亮】
http://www.kawa.net/
YAPC::NA 2008 in Chicago
Tuesday, 17 June 2008

Keyboard operation

[→] next
[←] prev
[↑] index

The slides use JavaScript-based "S6" presentation tool created by amachang.

§1
Introduction

(Q) Do you know how many APIs exist in the world?

programmableweb.com said

780 APIs! (June 17, 2008)
Roughly 0.8 new API everyday

(Q) Do you know how many APIs Google provides?

Counting Google APIs

1. access http://code.google.com/more/
2. enter following code to browser's location bar

javascript:{alert(CODESITE_productDictionary.products.keys.length)}

One-click translation

Demo: Good morning!

The slides are translatable for 24 languages with Google AJAX Language API

This translation could be better than none.

TODAY'S AGENDA

Introduction - Counting APIs
Who's Kawasaki?
Web services in Japan
WSST - main topic

§2
Who's Kawasaki?

Yusuke Kawasaki

川﨑　有亮

I came from Japan

日本から来ました。

Recruit Co., Ltd.

Gold sponsor of YAPC::Asia 2008 Tokyo
http://recruit.co.jp/

Media Technology Labs

Recruit's institute for research and development.
http://mtl.recruit.co.jp/

川﨑有亮 loves

Perl♥ & YAPC♥
JavaScript♥
Xiaolongbao♥

More detail: http://www.kawa.net/

CPAN Author

XML::TreePP

use XML::TreePP;
my $tpp = XML::TreePP->new();
my $url = "http://use.perl.org/index.rss";
my $tree = $tpp->parsehttp( GET => $url );
print $tree->{"rdf:RDF"}->{channel}->{title}, "\n";
print $tree->{"rdf:RDF"}->{channel}->{link}, "\n";

XML and Perl object mapping. (hash/array/scalar)
Pure Perl, fast and less dependencies to handle XML.
http://www.kawa.net/works/perl/treepp/treepp-e.html
Since 2006.02.20

XML::FeedPP

my $source = 'http://use.perl.org/index.rss';
my $feed = XML::FeedPP->new( $source );
print "Title: ", $feed->title(), "\n";
print "Date: ", $feed->pubDate(), "\n";
foreach my $item ( $feed->get_item() ) {
    print "URL: ", $item->link(), "\n";
    print "Title: ", $item->title(), "\n";
}

Pure Perl and easy to handle feeds without heavy dependencies.
http://www.kawa.net/works/perl/feedpp/feedpp-e.html
Since 2006.02.24

XML::FeedPP, cont.

my $feed = XML::FeedPP::Atom->new();
$feed->merge( "rss-2.xml" );     # RSS 2.0
$feed->merge( "rss-1.rdf" );     # RSS 1.0 (RDF)
$feed->merge( "atom-1.xml" );    # Atom 0.3/1.0

my $item = $feed->add_item( "http://search.cpan.org/~kawasaki/XML-TreePP-0.02" );
$item->title( "Pure Perl implementation for parsing/writing xml file" );
$item->pubDate( "2006-02-23T14:43:43+09:00" );

$feed->to_file( "index.rdf" );

RSS 2.0, RSS 1.0 (RDF), Atom 0.3 and 1.0 are supported.

§3
Web services in Japan

Recruit provides 14 kinds of APIs

Open data APIs for EC, car, local search, human resource, gourmet coupon, traveling, school, etc.
http://webservice.recruit.co.jp/

Mashup Award 4th

A largest contest for web application development in Japan.
1st prize: JPY￥1,000,000! (≒US$10K) http://mashupaward.jp/english/

38 Cosponsors (100+ APIs)

Most of web services are based on POX.
(POX = Plain Old XML)

<results>

  <api_version>1.00</api_version>
  <results_available>8629</results_available>
  <results_returned>10</results_returned>
  <results_start>1</results_start>
  <usedcar>

  <id>CU0001829778</id>
  <brand>
    <code>LE</code>
    <name>LEXUS</name>
  </brand>
  <model>SC</model>

  <grade>430</grade>
  <price>5300000</price>
  <desc>HDD navigation TV</desc>
  <body>
    <code>O</code>

    <name>Open</name>
  </body>

Recruit Web Service UI Library

JavaScript libraries to handle user interface elements on websites.

Mashup Example

Google Maps API＋Hotpepper Web Service
It took just two hours to read spec sheets and write this code!

UI Library Usage

<script type="text/javascript" src="js/jquery-1.2.3.js"></script>
<script type="text/javascript" src="rui/recruit.ui.js"></script>
<script type="text/javascript" src="rui/hotpepper.ui.js"></script>
<script type="text/javascript">
    function hpp_init (){
        Recruit.UI.key = '********';
        new HotPepper.UI.Places.Pulldown();
        new HotPepper.UI.Genre.Pulldown();
    }
    $(hpp_init);
</script>
<form id="hpp_sample" action="#">
Area: <select id="hpp-large-service-area-sel"></select>
<select id="hpp-service-area-sel"></select>
<select id="hpp-middle-area-sel"></select>
<select id="hpp-small-area-sel"></select><br>
Food genre: <select id="hpp-genre-sel"></select>
<input type="submit" value="Search" />
</form>

§4
WSST

(Q) How long do you need to write a Perl module? (in general)

2 months?
2 weeks?
2 days?

* Ingy took about only two hours to write Vroom.pm for OSDC.TW in Taipei, but it's an exception.

Our solution

WSS - WebService Specification Schema based on YAML

WSST - Tools to generate well tested Perl module by WSS

WSST needs only two seconds to write a Perl module!

Write once, get multiple libraries

In addition to Perl modules, we (will) support PHP, Python, Ruby, ActionScript etc. by one schema.

How many libraries required?

14 web services
X
5 languages
||
70 libraries (!?)

It's difficult for us to keep maintaining such large numbers of libraries by our hands!
Auto-library-generation is good way to do.

Perl module usage

use WebService::Recruit::CarSensor;

my $service = WebService::Recruit::CarSensor->new();
my $param = {
    'key' => 'XXXX',
    'pref' => '13',      # Tokyo
};
my $res = $service->usedcar( %$param );
my $root = $res->root;
my $list = $root->usedcar;

foreach my $car ( @$list ) {
    print "Model: ", $car->model, "\n";
    print "Brand: ", $car->brand->name, "\n";
    print "Price: JPY ", $car->price, "\n";
    print "Shop:  ", $car->shop->name, "\n\n";
}

CarSensor web service provides market prices of used cars.

PHP library usage

<?php
  require_once 'Services/Recruit/CarSensor.php';
  $service = new Services_Recruit_CarSensor(array('key' => 'XXXX'));
  $res = $service->usedcar(array('pref' => 13));
  $usedcar = $res->getData();
?>

<?php foreach($usedcar->usedcar as $i) { ?>
<table>
<tr><th>Model</th>
<td><?php echo htmlspecialchars($i->model) ?></td>

</tr>
<tr><th>Price</th>
<td>JPY <?php echo htmlspecialchars($i->price) ?></td>
</tr>

</table>
<?php } ?>

WSST is now on CPAN

http://search.cpan.org/dist/WSST/

Diagram

WSS (YAML) example

company_name: Flickr
service_name: Test
version: 0.0.1
methods:
  - name: echo
    title: flickr.test.echo
    url: http://api.flickr.com/services/rest/
    params:
      - name: method
        require: true
        fixed: flickr.test.echo
      - name: api_key
        require: true
    return:
      name: rsp
      children:
        - name: stat
        - name: method
        - name: api_key

WSS (YAML) example, cont.

    error:
      name: rsp
      children:
        - name: stat
        - name: err 
          children:
            - name: code
            - name: msg
    tests:
      - type: good
        params:
          api_key: $FLICKR_API_KEY
      - type: error
        params:
          api_key: invalid_api_key

Templates files for Template.pm

perl/lib/WebService/{company_name}/inc_webservice_method.tmpl
perl/lib/WebService/{company_name}/{service_name}/Base.pm.tmpl
perl/lib/WebService/{company_name}/{service_name}/{method.class_name}.pm.tmpl
perl/lib/WebService/{company_name}/{service_name}.pm.tmpl
perl/make-dist.sh.tmpl
perl/Makefile.PL.tmpl
perl/README.tmpl
perl/t/00_pod.t.tmpl
perl/t/01_new.t.tmpl
perl/t/02_new_methods.t.tmpl
perl/t/03_setget.t.tmpl
perl/t/inc_test_webservice_method.tmpl
perl/t/{test_seq}_Method_{method.class_name}.t.tmpl
perl/t/{test_seq}_{service_name}.t.tmpl

{method.class_name}.pm.tmpl

package [% package_name %]::[% method.class_name %];

use strict;
use base qw( [% package_name %]::Base );
use vars qw( $VERSION );
use Class::Accessor::Fast;
use Class::Accessor::Children::Fast;

$VERSION = '[% version %]';

=head1 NAME

[% package_name %]::[% method.class_name %] - [% title %] "[% method.name %]" API

=head1 SYNOPSIS

    use [% package_name %];
    
    my $service = [% package_name %]->new();
    
    my $param = {
[% FOREACH key = sort_keys(method.first_good_test.params) -%]
        '[% key %]' => [% method.first_good_test.params.$key | $env_param %],
[% END -%]
    };
    my $res = $service->[% method.interface_name %]( %$param );
    my $root = $res->root;
[% FOREACH node = tree_to_array(method.return) -%]
[% LAST IF loop.index > 6 -%]
[% NEXT IF node.depth == 1 -%]
    printf("[% node.name %]: %s\n", $root[% node_access(node) %]);
[% END -%]
    print "...\n";

Generating Perl module

$ wsst generate flickr-test.yml -l perl -o output
[perl]
output/perl/lib/WebService/Flickr/Test.pm
output/perl/lib/WebService/Flickr/Test/Base.pm
output/perl/lib/WebService/Flickr/Test/Echo.pm
output/perl/lib/WebService/Flickr/Test/Login.pm
output/perl/Makefile.PL
output/perl/README
output/perl/make-dist.sh
output/perl/t/00_pod.t
output/perl/t/01_new.t
output/perl/t/02_new_methods.t
output/perl/t/03_setget.t
output/perl/t/04_Test.t
output/perl/t/05_Method_Echo.t
output/perl/t/06_Method_Login.t

It takes roughly two seconds to generate a module.

Generating PHP library

$ wsst generate flickr-test.yml -l php -o output
[php]
output/php/package.xml
output/php/Test.php
output/php/Test/Factory.php
output/php/Test/Method/Echo.php
output/php/Test/Method/Login.php
output/php/Test/Method.php
output/php/Test/Request.php
output/php/Test/Request/Default.php
output/php/Test/Response.php
output/php/Test/Response/Default.php
output/php/docs/examples/echo.php
output/php/docs/examples/login.php
output/php/tests/AllTests.php
output/php/tests/Services/Flickr/Test/FactoryTest.php
output/php/tests/Services/Flickr/Test/Method/EchoTest.php
output/php/tests/Services/Flickr/Test/Method/LoginTest.php
output/php/tests/Services/Flickr/Test/Request/DefaultTest.php
output/php/tests/Services/Flickr/Test/RequestTest.php
output/php/tests/Services/Flickr/Test/Response/DefaultTest.php
output/php/tests/Services/Flickr/Test/ResponseTest.php
output/php/tests/Services/Flickr/TestTest.php

WebService::Simple::Cabinet

  use WebService::Simple::Cabinet;
  my $flickr = WebService::Simple::Cabinet->new(
      {
          global => {
              name     => 'flickr',
              package  => 'Flickr',
              base_url => 'http://api.flickr.com/services/rest/',
              params   => {
                  api_key => undef,
              },
          },
          method => [
              {
                  name   => 'echo',
                  params => {
                      method => 'flickr.test.echo',
                      name   => undef,
                  },
                  options => {},
              },
          ],
      },
      api_key => 'your_api_key',
  );
  my $res_xml = $flickr->echo( name => 'echo data' );
  print $res_xml->{name};

is yet another solution developed in CodeRepos.

Acknowledgements

Mitsuhisa Oshikawa
Co-author of WSST
Hitoshi Amano (a.k.a. amachang)
S6 presention tool's author

Thank you.

Feel free to contact me:
u-suke [at] kawa.net
http://www.kawa.net/