Nick is slamming random keys with oversized mits on

Friday, June 27, 2014

web nerdz

Subject: web nerdz

note: originally emailed 06/27/2014 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...

Many moons ago, I wrote the "http fun with telnet" post.

I think it needs to be revisited due to the renewed interest in 'standardized' REST calls (what web nerds say, "the way it was meant to be done")...

PUT vs POST

First, there's usually been some confusion with how REST calls are used --- in particular PUT vs POST.

Here are examples of what I mean:

http://stackoverflow.com/questions/4573305/rest-api-why-use-put-delete-post-get

A very simplistic implementation of REST could use the following CRUD mapping:

Create -> Post
Read -> Get
Update -> Put
Delete -> Delete

http://jcalcote.wordpress.com/2008/10/16/put-or-post-the-rest-of-the-story/

So often, developers will try to correlate these two concepts -- CRUD and REST -- using a one-to-one mapping of verbs from the two spaces, like this:

Create = PUT

Retrieve = GET

Update = POST

Delete = DELETE

But, really, a create/updating request should be thought of 'same thing'.

http://en.wikipedia.org/wiki/Create,_read,_update_and_delete

Operation		HTTP
Create		PUT / POST
Read (Retrieve)		GET
Update (Modify)		PUT / PATCH
Delete (Destroy)		DELETE

Note: PUT and DELETE are normally not enabled on all web servers

Server side processing

Because PUT and DELETE are quite dangerous if not controlled, server side processing (PHP, ASP, etc.) handled much of the PUT/DELETE (i.e. create, update, destroy) processes by incorporating them with AJAXy APIs (normally done with GET and POST only).

Let's see an example of the "server side processing" that a lot of people are used to:

http://webserver/account/?action=update&foo=bar
http://webserver/account/?action=delete&foo=bar
http://webserver/account/?action=upload -- POST with binary or text data
etc.

Here, the action parameter is descriptive, to the point and clear on intention.

I do realize that the URL "parameters" are much more descriptive than 'PUT'...

But, you'll need to see what is 'implied' when using GET/PUT/POST...

Idempotent Retries

There's a very important concept for web server technologies that needs to be understood before continuing

http://stackoverflow.com/questions/1077412/what-is-an-idempotent-operation
In other words, sometimes:

the web connection breaks
the browser mis-behaves or
user likes to hit the reload button

The web server needs to respond to the request as if it (the request) had been done only once

Think of this as self-recovery

Pretty URLs

A lot of web nerds want to slim down the URL so that search engines can "crawl" your web site with impunity.

Anyways, back to the implied usage of GET/PUT/POST/DELETE (what they call HTTP verbs). Let's take a look at an example URI:

http://webserver/account/nick

GET would return my details
PUT could update my details
A first time POST would create my account
And of course, DELETE would disable or nuke my account

Note that each of the methods does different things while the URL stays the same

You can still use server side processing to handle this

For example, in: http://webserver/account/

The index page has been setup as "app.php"

E.g. http://webserver/account/.htaccess contains:

DirectoryIndex app.php

app.php will process (for example) /account/nick as the ARGs for the script (and any POST or URL parameters) to handle the request

you can look at (http_fun_with_telnet) and add handlers for PUT and DELETE to complete this homework assignment

Tuesday, June 24, 2014

json-ld (part 3 of 3)

Subject: json-ld (part 3 of 3)

note: originally emailed 06/24/2014 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...
series: part 1 & part 2

Let me first say, I can see why they want all of this stuff:

xmlns:
RDF
JSON-LD
And finally -- from the only [usable] JSON-LD implementation available so far [that I know of so far]

a Hydra powered web API via symfony

This is the java nerds sticking their dirty fingers with a kung-fu death grip in the web world trying to make themselves still relevant.

Anyways, here's a HOWTO on getting "a" webby API thingie working with this JSON-LD stuff...

In a nutt shell
First, you need to know that Hydra is installed via "a" PHP package manger called Symfony

If you are interested in learning about Symfony, here's the brain dead simple howto:

http://symfony.com/doc/current/quick_tour/the_big_picture.html

Note: this installs locally and can be safely deleted

Just note that will require a running DB (e.g. mysql)

Now, install the sample hydra demo (so you have a working version to play with)

https://github.com/lanthaler/sfHydraDemoApp
Part 1 [of the demo] should look familiar if you did the crash course above
When you get to part 3 [of the demo]: pay attention to the last line:

"Please note that it will return JSON-LD so you might wanna access it using cURL or the Hydra Console"

Then there's "HydraBundle" -- which I'm guessing is a way to shove this Hydra server in your Symfony project.

https://github.com/lanthaler/HydraBundle

My analysis
Anyways, the Hydra demo implementation has confirmed my suspicion -- the webserver (or whatever RESTy box) needs to handle the URI path (as the parameter to your "handler")

E.g. http://webserver/account/foo/bar
In this case, the following does not actually exist on the "web server":

account/foo/bar
account/foo
account

and yes, the webserver walks those paths first

to see why, check out http://wiki.nginx.org/Pitfalls

"Passing Uncontrolled Requests to PHP"
Nick's mad HACK tip of the day!

finally, (in this case) a "index" file -- at http://webserver/ would take over

the demos have this file as "app.php"

this file takes and read the URI path and processes it by (depending on the php implementation) walk the "path" as the parameters for the request

so, http://webserver/app.php would process [ account/foo/bar ] and do with it as necessary

Side note: I've never used Symfony in the past -- and after this little exercise, I don't think I'll ever use it in the future...

Really? [A] java build system that compiles PHP code to include a bunch of other PHP frame work code...
And the "compiled" PHP code isn't even optimized -- cached, yes, but the number of files "loaded" is still HUGE
Yes, it's nice that it automates some steps, does some checks, it looks pretty... but wowzers -- it's big and this code is supposed to be called every time there's a REST request?

To understand how the REST URI is mapped to the handler, read this:

http://williamdurand.fr/2012/08/02/rest-apis-with-symfony2-the-right-way/
Basically, it's under app/config/*.yml

Security.yml is a good start

Note: This is part is from symphony -- hydra demo is just returning the json-ld response...

Personally, this is way too much code baggage for something that should be much simpler to ... implement?

Extra
And, since this was mentioned in today's meeting, if you've never used this before, BOOKMARK THIS:

http://jsbeautifier.org/

Shove your SUPER LONG LINE here and press the magic "beautify" button

Thursday, June 19, 2014

json-ld (part 2 of 3)

Subject: json-ld (part 2 of 3)

note: originally emailed 06/19/2014 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...
series: part 1 & part 3

This message is primarily for the TO: peeps. But some of you have asked about json-ld before and this would benefit everyone here for future reference.

Epiphany:
While reading up on some (computer) graphics articles -- I found out (in a round-about way) what ~~linked-data~~ (um, actually) the * description languages would be good for.

Not so fast:
First, why I still think it's not needed (in the context of javascript's json handler) -- it's unused information (you'll see this in a sec).

For more info on why I don't like json-ld (just like why I don't like VM's) -- it's heavy with a lot of un-needed cruff (you can read more about it from the email thread below).

Again, I'm coming from a javascript coder [for a project with restricted and security considerations] point of view...

So, what is it good for?
It (linked-data/interface description language/data format description language/etc.) is good for:

binary only data (i.e. structs/class)

getting transported between different programming languages
which might be running on different architecture (endian-ness, 32 vs 64 bits, etc.)

these (the data payload) are quite specific and can lead to catastrophic behavior if not handled properly

Examples of this type of mechanisms are:

ZeroC's Ice
Apache's Thrift
Google's Protocol Buffer
etc...

What does all that mean?
Basically, it's good for defining your data structs without (sort of) knowing anything about them.

Here's what I mean. Back to the graphics article I mentioned before. Blender (3D) is an open source modeler that had its beginnings 20 years ago.

http://code.blender.org/index.php/2013/12/how-blender-started-twenty-years-ago/

The creator of that program made his binary data files with "encoded types for the entire internal structure" -- in other words, self-describing and included in the data file. The intention was to make it backwards compatible knowing that future versions of the program will always require inevitable changes to the data file format.

http://www.blendernation.com/2008/12/01/blender-dna-rna-and-backward-compatibility/

Blender DNA, RNA and extreme backward compatibility
"This makes .blend files still readable, even when saved in 1.00"

This blew my mind.

20 years of the application -- and you can still load the data file that was made from v1.00

Not sure if a v2.71 (current version) data file would load on the v1.00 application, but you get the gist

Here are details on how this is done (just in case you're interested in how it was done)

Why strike ~~linked-data~~ and call out the * description languages
Back to "unused information" I mentioned earlier, [for our system] -- the LD stuff will NEVER be used (I can pretty much say that now with 99.9% certainty).

C/C++/java/etc. however, would HUGELY benefit from a self-describing struct/class object.

So when I started to play with:

http://json-ld.org/playground/

I said to myself, "this is interesting -- but what is this good for?"" -- I even said ([in part 1]):

this terse way of representing all of the data [is] highly error prone if written by humans -- this will need to be generated with tools just to be safe

The playground only showed results for JSON-LD and an RDF item.

But, there seems to be a number of possible representation for the same dataset:

http://rdf-translator.appspot.com/
- Raw data: ( CSML | CSV )
- RDF ( N-Triples | N3/Turtle | JSON | XML )
- OData ( Atom | JSON )
- Microdata ( JSON | HTML)
- JSON-LD

I thought to myself, there has got to be a better way to create this (remember: highly error prone if written by humans)

Then, the following showed up in [redacted] (today in fact!):

[link redacted]
They mentioned hive-json

This showed how a json ld-ish output was created from (in this case) the apache hive schema

So, out of curiosity, I thought if there were other ways to help write json-ld via converters?

Here's what I found:

http://jsonschema.net/reboot/#/

Write out your 'sample' json data
Click on the "schemarize" button it to generate the starter "schema"
Use the schema to help write the json-ld definitions

There were other resources I scanned through -- not really useful by itself, but helpful to figure out this linked-data stuff

http://www.cambridgesemantics.coam/semantic-university/sparql-by-example

Wednesday, May 28, 2014

json-ld (part 1 of 3)

Subject: json-ld (part 1 of 3)

note: originally emailed 05/28/2014 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...
series: part 2 & part 3

[Boss,]

Over the weekend, I looked into more about the JSON-LD stuff -- I get what it's doing -- but, we [are not] building a social network nor do we want dynamically generated 'queries' in areas that should be 'sanitized' (input validated, [...] etc.).

That said, the inventor of JSON-LD complained about a lot of things and the reason for creating JSON-LD (but hated on quad store a.k.a. named-graph).

http://manu.sporny.org/2014/json-ld-origins-2/

I'll getting to the "sparkle" part in just a bit. But, here's a sneak peek:

http://en.wikipedia.org/wiki/Named_graph

Anyways, if you ever read any of Microsoft's HTML exporter from their Word app you would see things like:

<html xmlns:v="urn:schemas-microsoft-com:vml"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:w="urn:schemas-microsoft-com:office:word"
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml"
xmlns="http://www.w3.org/TR/REC-html40">
<head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><title></title>
<style>
...

See all those xmlns: items?

Those are never used outside of your work, except for tracking purposes outside of your company. To "help" them figure out what you're document "means" instead of just a bunch of "content". This is what is known as the RDF annotation. RDF stands for Resource Description Format -- basically meta data for your content.

So, going back to JSON-LD: they want to transform something like this:

http://XYZ/ajax/api/path.php?user=name1
That would return something like this:

{ "result" : "pass", "balance" : "111111" }

To returning something like this:

{
    "@context": {
        "name": "http://xmlns.com/foaf/0.1/name",
        "balance": "http://json-ld-server/rdf/0.1/name",
    },
    "name": "X name1",
    "balance": "111111"
}

In other words, you need to describe what the data is -- instead of just dumping it.

Do note, this will also mean there's no stopping them from changing the simple ajax request of:

http://XYZ/ajax/api/path.php?user=name1

To something more convoluted like:

POST: http://json-ld-server/

POSTDATA:

{
    "@context": {
        "name": "http://xmlns.com/foaf/0.1/name",
        "query": "http://json-ld-server/rdf/0.1/query",
    },
    "name": "X name1",
    "query": "balance"
}

Again, all meant for systems outside of ours to help them understand our system (i.e. route around/through our data)...

Finally, SPARQL (pronounced -- sparkle) can be seen like this: S(PAR)QL -- which stands for: SPARQL Protocol and RDF Query Language

This allows you to do all things SQL (CREATE, INSERT, UPDATE, DROP, etc.) with this terse way of representing all of the data (highly error prone if written by humans -- this will need to be generated with tools just to be safe)
The named-graphic link above earlier, shows some examples -- but it's still means this needs someone to write the query... this is all duplicated work...

Tuesday, February 11, 2014

[ containers ]

Subject: [ containers ]
12/17/2012 06:21 PM

note: originally emailed 02/11/2014 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...

XXX mentions this way up the [internal] thread:

"we see open systems that move away from the monolithic closed stack (old way of delivering transactional systems) as the way forward." ...
"we see products such as the [abc] product converging with [xyz] POS solutions that allow us to extend our networks to open new and interesting sales channels (and other services such as promotions/advertising). it is a really exciting time."

I like the direction this is going.

It is no secret for my absolute disdain for VMware and those type of giant images (monolithic -- full OS on an image). Don't get me wrong -- I love LIVE distros (very flexible -- more importantly, runs on the bare metal) -- I just hate full blown 'images' that runs in a VM (too slow).

That said, projects like the docker *IS* the future of server/[software deployment] (fast and easy). <snip>

(Update: TODO: add link to unikernel post)

Here are some links for the OpenStack environment:

https://www.docker.io/learn_more/ or https://www.docker.io/the_whole_story/

See the part of Containers vs VMs

https://www.openstack.org/software/

Note: OpenStack is a standard/concept – not a downloadable project – those are provided by the vendors/community. [An example] community project is:

http://devstack.org/

http://linuxcontainers.org/

This is what makes a lot of this possible.
It's like chroot (jailed [filesystem] root) with rocket engines that blows away the competition

Sent: 12/17/2012 06:21 PM
Subject: [ server 1 & server 2]

Wow, it took about 2 hours to dance with vmware.

I would like to re-iterate my absolute detestation for OS-level virtual-machines.

In fact, just like my prediction of [xyz] going away -- I am going to say by the time we hand off [abc], it will NOT be a VMware image.

(I could NOT believe how LONG it took to make windows [change from] a static IP to dynamic and vice versa -- did this on the DB VM and was shocked at how long it took windows [on vmware] to complete that task -- it was faster to reboot the [image] instead of waiting for it to complete [the manual change over].)

<snip>

Monday, February 10, 2014

webGL supplemental

Subject: webGL supplemental

note: originally emailed 02/10/2014 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...

I forgot that I sent myself some links to write up about last weekend. And over this past weekend I found more stuff. < SNIP >

webGL traction

http://developer.amd.com/community/blog/2013/08/12/webgl-craze-at-siggraph-2013/

An article on the 'WebGL craze' from GDC last year

http://learningwebgl.com/blog/

A giant collection of links to articles and projects about ... webGL

http://webglstats.com/

A cool page showing the browsers supporting webGL's features and the rapid adoption of GL extensions in the past handful of months

https://plus.google.com/+DonOlmstead/posts/Mzy6VEAwHaa
https://plus.google.com/+DonOlmstead/posts/5akNbY6A15J

Sony's PS4 runs webGL for a bunch of its UX

3D modeler, animation and scene editor in a browser

http://exocortex.com/blog/claraio_open_beta

"3DStudio Max in a browser"
http://clara.io/

https://github.com/idflood/ThreeNodes.js

node editor -- this is fun to use

http://errolschwartz.com/projects/threescene/

scene editor (UI)

http://scenejs.org/

scene editor (coding)

http://stephaneginier.com/sculptgl/

sculpting app (school project -- not a product/release)

https://github.com/brianchirls/Seriously.js

video compositor

https://popcorn.webmaker.org/

NLE video editor

webGL development

http://benvanik.github.io/WebGL-Inspector/

Debugger like microsoft's PIX (Pixel Inspector)

http://www.gremedy.com/

For OpenGL -- but deserves honorable mention

webGL demos

There’s a pile out there:

http://acko.net/

Impressive modern examples

http://tutorialzine.com/2013/09/20-impressive-examples-for-learning-webgl-with-three-js/

Cool eye candy collections

http://www.awwwards.com/22-experimental-webgl-demo-examples.html

An older collection of examples

http://crypt-webgl.unigine.com/game.html

An engine (Unigine) I've seen in the past -- updated to run on webGL

http://www.mrdoob.com/

Too numerous to list, but definitely check out:

Sporel
HTML Editor (try changing the mesh.rotation value and see what happens)
Magic dust
GLSL Sandbox (another live editor)
Plane Deformations

Commercial projects

Translucent network
3 Dreams of Black
The Wilderness Downtown

Evan Wallace

http://evanw.github.io/glfx.js/

GL 2D effects

http://evanw.github.io/csg.js/

Code boolean geometry objects

http://evanw.github.io/webgl-filter/

Photoshop-ish effects

https://github.com/evanw/lightgl.js

GL lighting

[...] I think that’s enough for now…

Thursday, February 6, 2014

webGL

Subject: webGL

note: originally emailed 02/06/2014 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...

From: [name1]
The limiting factor (as expressed in the posts comments) is that it is depending on webGL – a standard/feature that has very little traction just yet.

From: [name2]
Ohhhhh Nick is going to freak out at you [name1] :)

All major desktop browser supports webGL. But, whether they are enabled (by default) is another question.

http://en.wikipedia.org/wiki/WebGL

webGL is even supported in IE11: (even though they had a kung-fu death grip on their Direct3D < SNIP >)

http://www.sitepoint.com/testing-limits-webgl-babylon-js-train-demo/

Most 'native' mobile browsers do not (out of the box) support webGL. But, again, all major mobile browsers do -- and one in particular requires to be jailbroken to be enabled or a ‘custom browser’ built with the iAd library.

I will say that mobile makers will try to delay webGL on their hardware as far away as possible due to the power hungry usage 3D visuals normally chews up compared to basic 2D render once and draw counterparts... anyways...

Now, as to whether there’s little traction or not:

http://www.chromeexperiments.com/webgl/

https://developer.mozilla.org/en-US/demos/tag/tech:webgl?sort=likes

My favorite is the banana bread demo

http://www.pouet.net/prodlist.php?platform[0]=JavaScript&page=1&order=thumbup

"the" demo scene collection
This section is all javascript -- a lot of them using the webGL for the performance and eye-candy goodies

Then, there's http://threejs.org/ -- this has become the webGL version of the openGL NeHe site

Their examples pages/link is quite large -- you will learn a lot to start your GFX programming skillz here...

[...] O'Reilly books [...] wrote this:

http://shop.oreilly.com/product/0636920024729.do
Almost a year and a half ago...

Little traction, huh...

I will make this prediction: (oh wait, I already did -- RE: News: Epic shows... & Unity Dropping Flash...)...

How about another prediction: let's see, I've already predicted that flash will die, I'm still waiting for java to die in the server world (to be over taken by NodeJS)...

How about this:

And [name2]'s email on the cool video of the live webGL shader editing:

https://hacks.mozilla.org/2013/11/live-editing-webgl-shaders-with-firefox-developer-tools/
This is KILLER!

... it’s almost a no brainer... there will be an browser based 3D modeler, animation and game editor/engine for both cloud or 'local' based development "webapp" – coming really really soon (I’m hoping for Blender 3D that runs in a browser).

Hmmm... that's not really a prediction. It’s inevitable.

Let's go along the lines of my last claim that: HTML5 will kill flash projects in [XXX]:

webGL will be used on [XXX product]s running off of webservers

true multi-game deployment
no custom OS requirements
easy on-the-fly upgrades
easy versioning deployment
etc. etc. etc.

Tuesday, November 5, 2013

Git branch naming

Subject: Git branch naming

note: originally emailed 10/30/2013 and sanitized for public consumption
sidenote: posting here so i can lead people here whenever they ask me about this stuff...

Sent by XXX:
> One thing I was thinking about since we are going to start using
> branches a lot more is that we start labeling branches versus tags. Say
> something like….'feature-branch' or 'feature-B' and
> 'ZZZ-1.1-T' or ZZZ-1.9-release'. This way it is easier to see
> which branches are tags and which branches are tags for release.

Don’t confuse a branch with a tag.

A tag is a ref like any other:

git checkout -b branchname tag

Which is a shortcut for

git branch branchname tag
git checkout branchname

“You cannot checkout a tag.”

http://www.gelato.unsw.edu.au/archives/git/0610/28708.html

Nick is slamming random keys with oversized mits on

Friday, June 27, 2014

web nerdz

Tuesday, June 24, 2014

json-ld (part 3 of 3)

Thursday, June 19, 2014

json-ld (part 2 of 3)

Wednesday, May 28, 2014

json-ld (part 1 of 3)

Tuesday, February 11, 2014

[ containers ]

Monday, February 10, 2014

webGL supplemental

Thursday, February 6, 2014

webGL

Tuesday, November 5, 2013

Git branch naming

Blog Archive

nick's shared items

s'more interesting reads

About Me

Nick is slamming random keys with oversized mits on

Friday, June 27, 2014

Tuesday, June 24, 2014

Thursday, June 19, 2014

Wednesday, May 28, 2014

Tuesday, February 11, 2014

Monday, February 10, 2014

Thursday, February 6, 2014

Tuesday, November 5, 2013

Subscribe To

Blog Archive

nick's shared items

s'more interesting reads

About Me