about passing structured data to flash

[Musicman]

I am using the term structured data for arrays, objects, arrays of objects, ...
To put up a specific example, I will consider product details that might be returned from searching a web shop.
The product data will consist of:
product name
unit price
order code
image (optional)
related link (optional) - if we are selling radios, this might go to spare batteries
unit qty / weight (optional)
Note that this scheme is not flexible enough for selling e.g. shoes - you would have one product name, but perhaps a list of order codes for different sizes
The flash data should be created as an array of objects, like
data = [{prod:'xxx',price:1.11,orderno:123,image:'iii.jpg'},{prod:'abc',price:2.22,orderno:456}];

Method 1 - plain flash vars
Use: loadvariables, Loadvars.load, Loadvars.sendAndLoad
The data string sent from the server could read
prod1=xxx&price1=1.11&orderno1=123&image1=iii.jpg&prod2=abc&price2=2.22&orderno2=456&count
=2
Now, if you wanted to build the flash array from the data, you would have to wait for the variables (!) and then use some code like that

Code:

data = [];
for(n = 1 ; n <= Number(count) ; n++)
{       prod = _root['prod'+n];
        price = _root['price'+n];
        orderno = _root['orderno'+n];
        item = {prod:prod, price:price, orderno:orderno};
        image = _root['image'+n];
        if(typeof(image) != 'undefined')
                item.image = image;
        related = _root['related'+n];
        if(typeof(related) != 'undefined')
                item.related = related;
        data.push(item);
}

Pro: using urlencode, all characters can be transmitted easily - no special handling required
Pro: data format is easy to understand and verify
Con: all data are received as strings
Con: extending data set to complex structures or multi-dimensional arrays may not be easy - consider data names like xyz13_21
Con: creates lots of variables, so in fact the _root should be replaced by a dedicated MC receiving the vars

Method 2 - comma-separated strings
Use: loadvariables, Loadvars.load, Loadvars.sendAndLoad
The data string sent from the server could read
prods=xxx,1.11,123,iii.jpg,|abc,2.22,456,,
To read that data, you would again wait for data (!) and then use code like this

Code:

data = prods.split('|');
for(n = 0 ; n < data.length ; n++)
{       d = data[n].split(',');
        item = {prod:d[0], price:d[1], orderno:d[2]};
        if(d[3] != '')
                item.image = d[3];
        if(d[4] != '')
                item.related = d[4];
        data[n] = item;
}

Pro: small data size
Con: only works when separator chars do not occur within data; this may be hard to achieve, and there is no obvious way to include all characters
Con: all data are received as strings
Con: no easy and fool-proof way to extend the data format - adding a new column to the data in the wrong place may go unnoticed
Con: at least FP5 got severe performance problems splitting those strings

Method 3 - XML
Use: XML.load, XML.sendAndLoad
The data sent from the server might read

Code:

<prods>
        <item>
                <prod>xxx</prod>
                <price>1.11</price>
                <orderno>123</orderno>
                <image>iii.jpg</image>
        </item>
        <item>
                <prod>abc</prod>
                <price>2.22</price>
                <orderno>456</orderno>
        </item>
</prods>

Once the data loads, some actionscript would be necessary to convert the received xml object into the target array
Pro: easy to read and verify; if you care to write a formal specification for the data, you can even use available tools to verify the output of the server script
Pro: easy to extend, data format handles complex structures nicely
Pro: flash client could be replaced by something else
Pro: arrays / objects on the server can more or less automatically be converted into xml and back through inclusion of canned code (distinction between a data item and an array holding exactly 1 item may be lost)
Con: more coding to create and parse the xml than other methods, if it has to be done individually
Con: FP5 did not handle xml parsing too well either

Method 4 - binary
Use: loadmovie
The data is not sent as readable text but rather as a movie consisting of scripts only.
Now, the flash player does not read actionscript source, like the
data = [{prod:'xxx',price:1.11,orderno:123,image:'iii.jpg'},{prod:'abc',price:2.22,orderno:456}];
specification, but only compiled actionscript. So for this method we need _something_ on the server that compiles actionscript. This could be MING since it includes a full AS compiler.
This could also be a php class or perl package that converts php or perl data into a movie
Sample MING code

Code:

<?
$m = new SWFMovie();
$m->add(new SWFAction("
data = [
{prod:'xxx',price:1.11,orderno:123,image:'iii.jpg'},
{prod:'abc',price:2.22,orderno:456}
];
"));
header('Content-type: application/x-shockwave-flash');
$m->output();
?>

If the data is derived from searching a database, that line of actionscript would have to
be created by the
server script first.
The above-mentioned php class works with arrays and provides methods to set a variable or
call a function, so code might be

Code:

<?
require "swfdata.php";
$data = array(
array(prod=>'xxx', price=>1.11, orderno=>123, image=>'iii.jpg'),
array(prod=>'abc', price=>2.22, orderno=>456)
);
$d = new swfdata();
$d->setvar('prods', $data);
$d->send();
?>

Creating an array from a database search seems to be somewhat easier than creating a string
Pro: no code required in receiving movie, the result data structure just "comes into life" when the transfer completes
Pro: data typing determined by server
Pro: full character set can be passed without any effort
Con: if any errors occur on the server, a movie transmitting just an error message has to be sent
Con: data is not human readable, so extra software is required to check that a server script is indeed sending correct data
Con: some idiot servers add "banners" to all script output, even to generated swf ... which certainly will not work - and may take ages to diagnose

Method 5 - binary AMF
Use: NetConnection.call
Yet another way of passing binary data, commonly called "Flash remoting".
Here the movie can send actionscript objects as a binary message, and the server returns a similar message. Like above, the data on the server would appear as an array or object in the server scripting language.
Sample application - flash part:

Code:

var nc = new NetConnection();
nc.connect(url);
var reply = {};
reply.onResult = function(result) {
        trace('result is '+result+' ('+typeof(result)+')');
};
nc.call('script1', reply, data1, data2); // here data1 and data2 are arbitrary flash data items

Sample application - server part:

Code:

<?
include "amfdata.php";  // parse request
$rqst = new amfdata();  // parse request
switch($rqst->fn)ipt1':
{       case 'script1':
                $data = $rqst->fnargs[0];/
                // process data herejekte/binvars/struc.txt lines 127-149/165 85%
                $data = array(-3.7, 4.5);
                $rqst->sendresult($data);
}
?>

Pro: passing data in either direction as actionscript objects - all the other methods utilized plain flash vars to send the request.
Pro: data typing determined by server
Pro: full character set can be passed without any effort
Con: if any errors occur on the server, a movie transmitting just an error message has to be sent
Con: undocumented format, typically requires a fairly pricey server solution and uses some not-so-common server languages like CF and hence does not lend itself to experimentation
Con: requires FP6
Con: data is not human readable, so extra software is required to check that a server script is indeed sending correct data - for the movie method above one of the available action script analyzers should do, there does not seem to be software for amf around.

So far, the latter method was only available together with a suitable server, now there is a partial php class as well.
Both binary scripts suffer from the fact that PHP does not consider 0xfefefefe as an integer

Musicman

[fh]

Hi musicman
Interesting article you wrote. As a programmer myself I believe that XML and binary transfer are the only two nice ways. Here you can transfer arrays/recordsets, and thereby transfer big amount of data in a standard way.
Some time ago I converted my websites to binary format/remoting, but did not find that the websites were much faster, so I went back to XML. Of course you could choose to put XML into the binary format!!. I did this several times and it works very good. Then inside Flash you can create functions that uses the XML object, ex. next/previous and it is very easy to build masks on the mc's at the same time the transfer can be on a binary format.
Again I found that the binary transfer had some problems using utf-8 and I had to code a lot of stuff. So now I use XML with php.


fh

[pitu]

Hello musicman,

Could you please explain about the 4-th method using loadmovie and php class - how can the data be accessed from flash. I tried:

 prods[0].prod      by example and it does not work..

. I can not use ming on my server so this solution is most adequate.

..anyways...Thank you for this most resourcefull post

[Musicman]

Hi,

this is  most likely a reference issue - the variable is set inside the loaded movie, so you would need to target the data via its instance name

Musicman

[pitu]

 Hello again

Thank you for your remark ) It made me re examen my code ...I was indeed targeting the movie where I thought it would load...but an actionscript error did not load it there ))  So now it works fine.

I also needed to save the file instead of outputting it so here is what I did:

added this in the code that creates the swf:

Code:

$fileName="somefile.swf";
if (! file_exists($fileName)) {touch($fileName); chmod($fileName,0644); }
$d->save($fileName);

and added a "save" function in the "swfdata.php" class like this:

Code:

function save($fileName)
   {   $this->actions .= "\0";
      $len = strlen($this->actions);
      if($len > 63)
         $this->actions = pack("vV", (12 << 6) + 63, $len) . $this->actions;
      else
         $this->actions = pack("v", (12 << 6) + $len) . $this->actions;
      $sz = 8 + strlen($this->hdr2) + strlen($this->bgcol) + strlen($this->actions) + strlen($this->end);
      //header('Content-type: application/x-shockwave-flash');
      
      //if (! file_exists($fileName)) {touch($fileName); chmod($fileName,0644); }
      $handler=fopen($fileName, 'w');
      $content= $this->hdr1 . pack("V", $sz) . $this->hdr2 . $this->bgcol . $this->actions . $this->end;
      fwrite ($handler, $content );
      fclose($handler);
   }

[starace]

Musicman,

This is one awesome post, nobody has gone through all the methods that you have gone through here. Hat's off to you.

I have a binary load/save working as per your example, but I have hit one pitfall. That is the character "Ë" that won't pass from the loaded movie. Everything else is working fabulous, the php script is working to create a swf, and when I parse the created swf I can see the "Ë", but it just doesn't show up when I load it.

Is this a bug in Flash, or am I doing something wrong?

To test this I even created a swf in flash with just a textfield with, let's say "Work Ë".

I create another movie to load it into and the funny thing is that the "Work" populates but the "Ë" is missing.

Do you have any ideas on what might be wrong. I've spent alot of hours getting this to work and I'm very excited with your techniques and would hate to have to go in another direction now.  This is awesome, I even have the php generated swf being saved to a blob field in mysql.


Thanks again, and GREAT post.

Starace

[starace]

After a few more tests it seems like I can go as far as ascii 191 then it chokes. Any ideas how I could extend this to include more characters?

Thanks again.

Starace

[Musicman]

Hi,

are you sending 1 character, or a utf8 pair? Unless you mess with codepage, flash should accept utf8

Musicman

[starace]

Musicman,

Thanks for the reply.

In my testing I was using the following and it was working okay, but had that problem with special characters.

$data1 = utf8_decode($_POST["name"]);

After your response, I removed the utf8_decode comment and it seems to be a working.

$data1 = ($_POST["name"]);


I cannot state emphatically enough how innovative and awesome this post is on saving binary data.

I'm going to have a movie that needs to save a hundered or so small strings and movie clip parameters, like true, false, posx, posy, and this is gonna be the answer..... BIGTIME!!!

XML was going to be my next try, but I don't think I could have handled the parsing.

A comma seperated was going to be my fallback solution.

Thanks again for the absolutely AMAZING and WORKING approach!

Starace