Okay - some people have been making some rather brazen claims about the ability to crack a PDF document which uses the AES 128 bit key to encrypt the content. I am not sure if they are truly psychic (one plausible way to crack AES), but doing such using a brute force attack is probably not going to work without a bit of luck. To understand how complex AES is, let's explore how the key is derived and what it means to "break" or "crack" encryption.
The AES standard has a fixed block size of 128 bits and a key size of either 128, 192 or 256 bits. The key is expanded using Rijndael's key schedule in which most of AES calculations are done in a special finite field. Operates on a 4 x —4 array of bytes (the State) and uses four distinct steps to encrypt.
1. AddRoundKey - each byte of the state is combined with the round key; each round key is derived from the cipher key using a key schedule (simple).
2. SubBytes - SubBytes is a non-linear substitution step where each byte is replaced with another according to a lookup table. In order to add non-linearity to the key, the Galois Field (GF) is used to derive the inverse function of 2 to the power of 8. This has been credited with keeping things unlinear.
3. ShiftRows - a transposition step where each row of the state is shifted cyclically a certain number of steps.
4. MixColumns - a mixing operation which operates on the columns of the state, combining the four bytes in each column using a linear transformation. This provides diffusion in the final cipher in conjunction with ShiftRows. Each column is treated as a polynomial over GF(28) and is then multiplied modulo x4 + 1 with a fixed polynomial c(x) = 3x3 + x2 + x + 2.
The final round replaces the MixColumns stage with another instance of AddRoundKey.
To give you an idea of the complexity of the resulting decryption process, to crack a 128 bit key would take approximately 3.4 x 10^38 guesses assuming the correct key was the last one you tried. To place this in perspective, is estimated that if a DES key generator were able to discover 1 DES key per second, it would take 149 thousand-billion (149 trillion) years to crack a single 128 bit AES key. As a side note, most physicists accept that the universe is approximately 20 billion years old.
As Homer Simpson would say - "D'oh!!!".
What people who are claiming to "break" AES have to use as a metric is a methodology which is anything faster than an exhaustive search for all the keys, also referred to as a "brute force" attack. There is another type of attack methodology called a "side channel attack which does not operate on the actual cipher but on mechanisms around it. There have been two claims of success in breaking AES using side channel attacks. You should note that both of these required access to an application running on the same machine.
Anyone still feel they can get the $500 I offered for cracking the document?
Monday, May 01, 2006
Friday, April 28, 2006
Working with XML data in Flex and Flash
I have been playing around with a new feature of Flex 2.0 (actually it is a new part of ActionScript 3.0). It is called E4X or ECMAScript for XML. It allows Flex developers to very simply and quickly write down and dirty bindings from XML data to Flex objects.
I will be teaching a 4 hour tutorial at the upcoming Geoweb in Vancouver July 24-28 2006. My good friends at Galdos host this conference and do a great job every year. This year the course will be a hands on developer course with Flex 2.0 and a ton of ActionScripting and even a finale of using the Flex API to Yahoo maps.
Okay – so I am not going to repeat the entire tutorial module here (it is not even complete), but I will share the *.mxml file with you. It is commented to be relatively clear what is going on. For clarity, I put the GML XML sample inline. To work with XML in ActionScript, you simply declare the XML data as an object, then you can use the various XPath like filters and queries against it. While it is not as robust as the full blown JAXB, it is very effective. Here is the sample file – just cut and paste it into your Flex 2.0 project and start playing around by un-commenting the various lines that demonstrate the functionality. The actual file can be downloaded from http://www.nickull.net/blogimages/GML.mxml to avoid retyping.
Enjoy!
I will be teaching a 4 hour tutorial at the upcoming Geoweb in Vancouver July 24-28 2006. My good friends at Galdos host this conference and do a great job every year. This year the course will be a hands on developer course with Flex 2.0 and a ton of ActionScripting and even a finale of using the Flex API to Yahoo maps.
Okay – so I am not going to repeat the entire tutorial module here (it is not even complete), but I will share the *.mxml file with you. It is commented to be relatively clear what is going on. For clarity, I put the GML XML sample inline. To work with XML in ActionScript, you simply declare the XML data as an object, then you can use the various XPath like filters and queries against it. While it is not as robust as the full blown JAXB, it is very effective. Here is the sample file – just cut and paste it into your Flex 2.0 project and start playing around by un-commenting the various lines that demonstrate the functionality. The actual file can be downloaded from http://www.nickull.net/blogimages/GML.mxml to avoid retyping.
Enjoy!
Friday, April 21, 2006
AJAX hammers servers?
I read a question posed by James Governor's blog and Tim Bray's always insightful response to it yet was left feeling unfulfilled. It lead me to this blog which provided some further information. Perhaps it is the jetlag or I just haven't abused enough caffeine today but I feel compelled to write more.
The author does not define several things which make his statements relatively un-confirmable. First of all, there is no such thing as a spec for the Web 2.0 so his claim that RIA's are a crucial aspect is a flawed assumption from a pragmatic standpoint, although somewhat orthogonal to the question asked.
So let's look at what AJAX does and why it is favored. In the past, if you wanted to make a webpage that displayed up to date information, you may have to force the page to refresh itself every few minutes or prompt the user to do this. Since HTTP is stateless, that means firing off an HTTP get() request every few minutes to retrieve the entire page. The easiest way to do this was to use the meta-refresh element of HTML 4.0 transitional. Since HTTP server are idempotent, they respond each time they get the request. Tim and I had a conversation about this back in 1999-2000 when he called it TAXI (the father of AJAX). The question posed was "wouldn't it be much more efficient to only refresh parts of a page rather than the whole page). Out of that necessity, AJAX was born. It was actually Microsoft that cemented it by putting the XMLHTTPRequest() object into IE.
Now let's separate business needs from technology. If you have a business requirement to provide current, up to date information to your end users via the internet, you are going to do it, regardless of the underlying technology and the costs on servers. From a pure pragmatic standpoint, one should want to do this in the most efficient manner possible. There are several models available to use.
One is that the server "pushes" information to clients when some event happens (perhaps a stock price changes). The problem with this is that it often means the server has to dispatch a large number of concurrent messages when that event occurs, even if the clients themselves are not requesting it. Suboptimal - this can cause server overload, scare small children and bruise fruit.
Second model is the client side pull. This usually makes more sense given the client controls the nature of the request frequency, although the services architects determine the content size and policies for requests. This makes it much easier on the server to balance its outward messages given not all clients are likely to request at the exact same time.
Reloading only part of a page or even just the data for that part of the page is much more efficient that having to load an entire page or the data and presentation aspects of a component of the page, therefore, I would state that AJAX (or other AJAXian type methodologies) are probably the most efficient way to handle the business requirements that are placed on the web today.
If you were to ban AJAX from the web, we would have to revert to full page reloads which would certainly be more bandwidth and processor intensive than AJAX enables.
My opinion - those who are online gamers, porn surfers, MP3 downloaders are all more likely to cause the scalability problems that AJAX developers. Carefully thought out architecture should be used where possible. AJAX solves more problems that it creates.
Duane
The author does not define several things which make his statements relatively un-confirmable. First of all, there is no such thing as a spec for the Web 2.0 so his claim that RIA's are a crucial aspect is a flawed assumption from a pragmatic standpoint, although somewhat orthogonal to the question asked.
So let's look at what AJAX does and why it is favored. In the past, if you wanted to make a webpage that displayed up to date information, you may have to force the page to refresh itself every few minutes or prompt the user to do this. Since HTTP is stateless, that means firing off an HTTP get() request every few minutes to retrieve the entire page. The easiest way to do this was to use the meta-refresh element of HTML 4.0 transitional. Since HTTP server are idempotent, they respond each time they get the request. Tim and I had a conversation about this back in 1999-2000 when he called it TAXI (the father of AJAX). The question posed was "wouldn't it be much more efficient to only refresh parts of a page rather than the whole page). Out of that necessity, AJAX was born. It was actually Microsoft that cemented it by putting the XMLHTTPRequest() object into IE.
Now let's separate business needs from technology. If you have a business requirement to provide current, up to date information to your end users via the internet, you are going to do it, regardless of the underlying technology and the costs on servers. From a pure pragmatic standpoint, one should want to do this in the most efficient manner possible. There are several models available to use.
One is that the server "pushes" information to clients when some event happens (perhaps a stock price changes). The problem with this is that it often means the server has to dispatch a large number of concurrent messages when that event occurs, even if the clients themselves are not requesting it. Suboptimal - this can cause server overload, scare small children and bruise fruit.
Second model is the client side pull. This usually makes more sense given the client controls the nature of the request frequency, although the services architects determine the content size and policies for requests. This makes it much easier on the server to balance its outward messages given not all clients are likely to request at the exact same time.
Reloading only part of a page or even just the data for that part of the page is much more efficient that having to load an entire page or the data and presentation aspects of a component of the page, therefore, I would state that AJAX (or other AJAXian type methodologies) are probably the most efficient way to handle the business requirements that are placed on the web today.
If you were to ban AJAX from the web, we would have to revert to full page reloads which would certainly be more bandwidth and processor intensive than AJAX enables.
My opinion - those who are online gamers, porn surfers, MP3 downloaders are all more likely to cause the scalability problems that AJAX developers. Carefully thought out architecture should be used where possible. AJAX solves more problems that it creates.
Duane
Wednesday, April 19, 2006
Circumventing PDF with gmail? Not!!
I was recently amused by reading a blog of a group who apparently
defeated PDF's DRM system by using GMail's "convert to HTML" option. I nearly fell off my chair when I read the claim " (it) works regardless of the files; usage restrictions..". Yes - under certain circumstances you can gain access to text or other components of a PDF document that has policy protection on it, but *only* if the person applying the policies set the policies to allow this type of access AND does not encrypt the PDF. Keep in mind that PDF is a completely free, open and available standard that anyone can implement. There are several third party SDK's to manipulate PDF documents. Before you read the blog above, it is extremely helpful to understand how the encryption and DRM mechanisms work.
In general, if you do not want someone other than the intended recipient to
view a PDF, you should encrypt it. By default, the encryption level for compatibility with Acrobat 5.0 and later is 128bit RC4. Encrypting the contents of a PDF with a strong key results in a situation where there is no way gmail or any other
application can crack it open by brute force. The PDF is turned into cipher text that is completely incomprehensible to anyone without the key to open it. I am so certain of this that I will provide $500 USD to the first person who can open this document within one year.
A person encrypting a PDF document has several options. First, you can determine the compatibility for earlier versions of Acrobat (5 , 6) or jump straight to Acrobat 7.0 and higher. If you select to encrypt it for Acrobat 7, the default level encryption method is AES, much harder (read = impossible) to crack using brute force.
You can also opt to encrypt all the document contents, or leave the metadata unencrypted. This is useful should you want to be able to have the document searchable in real time based on the metadata. Note the lower section of the screenshot above - by default, the box is checked to allow text access to the document. If you leave this selected, some PDF applications can access the text. If you don't want this, please de-select this option. After setting all of the options and pressing next, you will still be given a generic warning that certain non-Adobe products might not enforce this document's policies. Note that if you do not select "require a password to open the document", the usefulness of encrypting it is moot. Others will still not be able to copy the document by using the text copy tool or Control-C, but other means can be employed.
To summarize so far, Acrobat has DRM capabilities to limit the following interactions with documents
1. ability to disable printing
2. ability to disable cut and paste
3. ability to disable control printscreen
4. ability to disable local file saving
5. ability to disable local file saving
6. ability to disable accessibility
7. ability to make a document no longer exist
A person must comprehend the frame and scope of the intended use of each of
these and their built in restrictions. PDF's are like music - if you can
render it once, it is possible to capture it and render it again. Even if
we figured out a way to prevent all third party screen scraping software
from capturing what you see on a computer screen, someone who both has
access to the document for a single view AND intent to distribute it further can simply take adigital photo of their computer screen to circumvent all of these. There is simply no way to stop someone who is intent on doing this using 1-6 above.
Another methodology is available to place a dynamic watermark on the page, perhaps stating the users name and address in bold gray text across the document. This too can be defeated if one took a screen shot of the document and used a great tool like ... err "Adobe Photoshop" to take care of that nasty watermark. I am guessing the magic wand tool is your best friend here ;-)
So how can you protect a PDF? If you really want to make it secure and also
track the users interaction with it, you would be wise to use Adobe Policy
Server. The policy server uses a model of persistent DRM that follows the
document everywhere it goes. If you feel the document is out of control and
you want to stop it, you can simply "destroy" the document which will cause
it to fail to un-encrypt itself when someone opens it. Is there a way
around that? Sure - sneak into the office of the person who made the
policy, install a tiny pinhole camera near their desk and capture their
authentication.
See what I am getting at, no matter what you do, there is a way around it if
someone is really intent. The easier method is "social engineering" rather than brute force.
So here is a challenge. Take this document here (link to APS protected
document) and try to render it with gmail (or any other method). I will pay
$500 USD to the first person who can show me the un-encrypted content of this document within one year of this.
How I would do it? I would probably try to lure myself into providing a password to a site that offered me some form of membership and hope that I was rather lazy and used the same password for this document. D'oh!! Not gonna work - I typed a random phrase of about 13 characters to encrypt this using AES.
Good luck!
defeated PDF's DRM system by using GMail's "convert to HTML" option. I nearly fell off my chair when I read the claim " (it) works regardless of the files; usage restrictions..". Yes - under certain circumstances you can gain access to text or other components of a PDF document that has policy protection on it, but *only* if the person applying the policies set the policies to allow this type of access AND does not encrypt the PDF. Keep in mind that PDF is a completely free, open and available standard that anyone can implement. There are several third party SDK's to manipulate PDF documents. Before you read the blog above, it is extremely helpful to understand how the encryption and DRM mechanisms work.
In general, if you do not want someone other than the intended recipient to
view a PDF, you should encrypt it. By default, the encryption level for compatibility with Acrobat 5.0 and later is 128bit RC4. Encrypting the contents of a PDF with a strong key results in a situation where there is no way gmail or any other
application can crack it open by brute force. The PDF is turned into cipher text that is completely incomprehensible to anyone without the key to open it. I am so certain of this that I will provide $500 USD to the first person who can open this document within one year.
A person encrypting a PDF document has several options. First, you can determine the compatibility for earlier versions of Acrobat (5 , 6) or jump straight to Acrobat 7.0 and higher. If you select to encrypt it for Acrobat 7, the default level encryption method is AES, much harder (read = impossible) to crack using brute force.
You can also opt to encrypt all the document contents, or leave the metadata unencrypted. This is useful should you want to be able to have the document searchable in real time based on the metadata. Note the lower section of the screenshot above - by default, the box is checked to allow text access to the document. If you leave this selected, some PDF applications can access the text. If you don't want this, please de-select this option. After setting all of the options and pressing next, you will still be given a generic warning that certain non-Adobe products might not enforce this document's policies. Note that if you do not select "require a password to open the document", the usefulness of encrypting it is moot. Others will still not be able to copy the document by using the text copy tool or Control-C, but other means can be employed.
To summarize so far, Acrobat has DRM capabilities to limit the following interactions with documents
1. ability to disable printing
2. ability to disable cut and paste
3. ability to disable control printscreen
4. ability to disable local file saving
5. ability to disable local file saving
6. ability to disable accessibility
7. ability to make a document no longer exist
A person must comprehend the frame and scope of the intended use of each of
these and their built in restrictions. PDF's are like music - if you can
render it once, it is possible to capture it and render it again. Even if
we figured out a way to prevent all third party screen scraping software
from capturing what you see on a computer screen, someone who both has
access to the document for a single view AND intent to distribute it further can simply take adigital photo of their computer screen to circumvent all of these. There is simply no way to stop someone who is intent on doing this using 1-6 above.
Another methodology is available to place a dynamic watermark on the page, perhaps stating the users name and address in bold gray text across the document. This too can be defeated if one took a screen shot of the document and used a great tool like ... err "Adobe Photoshop" to take care of that nasty watermark. I am guessing the magic wand tool is your best friend here ;-)
So how can you protect a PDF? If you really want to make it secure and also
track the users interaction with it, you would be wise to use Adobe Policy
Server. The policy server uses a model of persistent DRM that follows the
document everywhere it goes. If you feel the document is out of control and
you want to stop it, you can simply "destroy" the document which will cause
it to fail to un-encrypt itself when someone opens it. Is there a way
around that? Sure - sneak into the office of the person who made the
policy, install a tiny pinhole camera near their desk and capture their
authentication.
See what I am getting at, no matter what you do, there is a way around it if
someone is really intent. The easier method is "social engineering" rather than brute force.
So here is a challenge. Take this document here (link to APS protected
document) and try to render it with gmail (or any other method). I will pay
$500 USD to the first person who can show me the un-encrypted content of this document within one year of this.
How I would do it? I would probably try to lure myself into providing a password to a site that offered me some form of membership and hope that I was rather lazy and used the same password for this document. D'oh!! Not gonna work - I typed a random phrase of about 13 characters to encrypt this using AES.
Good luck!
Tuesday, April 18, 2006
The Web 2.0
I now feel compelled to write in this subject after holding my breath and counting to 10 several times. I recently read YABA* about building a "Web 2.0 Meter". A not bad idea *if* you had some sort of defined criteria that those being judged could adhere to. Another site claims it is a Web 2.0 validator
Web 2.0 Validator.
Enough! I can even hear Homer Simpson saying “D’oh!!” when he thinks about this. Time to rant a bit from Logic 101. You cannot measure something by two independent “meters” without some distinct set of metrics around the subject. Sorry folks, it is that simple. I would like to point out that the Web 2.0 validator site had the sense to state the rules they use and that the gist of the article at Oreilly was not about the web 2.0 meter. It was a realization that the concepts we have come to “associate” with the Web 2.0 were really the Web 1.0’s original goals. GAH!!! I just had an unpleasant realization that now I am trying to quantify the Web 2.0. To solve this problem, I think we have to look to the past as well as the future (yeah yeah – so what does that rule out? Thinking about the exact present moment?). The Oreilly folks are pretty smart IMO so please don't take this as some petty stab - more of a friendly prod :-)
If “Web 2.0” is to be used as a catch all term for where we are going, let's put some substance behind it. If not, it will suffer the same symptoms as SOA and Web Services - both very meaningful to most people, just with differing semantics. In fact, the lack of clarity around SOA lead a group of almost 200 people to get together and write a formal Reference Model for SOA under the auspices of OASIS. Similarly, a group got together within W3C and worked on a Reference Architecture for Web Services. I am proud to state that I worked on both projects.
So what can be done to put substance in the Web 2.0?
1. Write an abstract reference model to show the components of the Web 2.0 (abstract); and
2. Create sets of high level abstract patterns, mid level patterns and low level idioms to illustrate what is really meant by the web 2.0; and
3. Create reference architecture (in plural and somewhat generic) for all components of the Web 2.0, describing their externally visible properties and relationships with other components.
An example would be to illustrate the syndication-subscription pattern using some architectural patterns template. The abstract notion is that subscribers notify a syndication component of their wish to receive content. When the syndication component has content it is ready to push out, it configures a list of recipients based on some criteria then proceeds to push the content out. A lower level idiom could show this implemented using Apache components, perhaps even with options for content formatting based on device, reliable messaging protocols, security and end user authentication with a persistent security model for the content itself.
The cool thing about this approach is that it still gives each and every implementer the freedom to make their own black box components whilst preserving a common layer of understanding. It also provides documentation about what is really meant, granted, those who cannot distinguish abstract from concrete may still be confused.
Of course to do this, you would require an architectural patterns meta model and template that allowed you to go from the very abstract to the very concrete, but I think I know where one is that can be donated to some organization.
Why should this be done? Simple – without this, “Web 2.0” is nothing more that a marketing term. Sure – several people will say “no – it means X and exactly X”, but the chances of Boolean Y = eval(personA.X == personB.X) evaluating to “1” in every instance is very low IMHO.
The Web 1.0, aka the “internet”, has achieved a common definition though. Even though it is not concisely written, there is general consensus on what a web server does, what the layered wire protocols do, how security works and how people interact with websites (via browsers). If someone says “this server is internet enabled”, people imply that it means it can take HTTP requests and return text in compliance with the requesters requirements.
Sorry – folks. I just don’t believe that the Web 2.0 will inherit an implied reference model the way the Web 1.0 did. The culprit for this is the Web 1.0 as it exists – it allows anyone, almost anywhere, to write what they think the Web 2.0 is and share it with others. Also, unlike the Web 1.0, the Web 2.0 is not mandatory. The basic components of the Web 1.0 such as HTTP, TCP/IP, SMTP, MIME, HTML etc all were mandatory, therefore it was fairly easy to draw a box and state – this is the Web 1.0. One could even through in some common non-mandatory components and still make a solid statement (example – scripting languages (ASP, VBScript, JavaScript, ActionScript plus CSS, XML et al).
A Reference Model for the Web 2.0 might want to declare some form of compliancy and conformancy statements. Such might be a weighted test of it could be a bar that you must pass. Regardless, before this exists, what is the point of building “validators” and “meters”. Harrumph – end of rant. Take it all with a grain of salt – the Oreilly folks are smart and I’m sure we’ll see something soon ;-)
If not – does anyone feel compelled to take a stab at a formal definition? I will gladly jump in the fray and donate my time to help.
*Yet Another Blog Article – in case you didn’t figure it out ;-)
Web 2.0 Validator.
Enough! I can even hear Homer Simpson saying “D’oh!!” when he thinks about this. Time to rant a bit from Logic 101. You cannot measure something by two independent “meters” without some distinct set of metrics around the subject. Sorry folks, it is that simple. I would like to point out that the Web 2.0 validator site had the sense to state the rules they use and that the gist of the article at Oreilly was not about the web 2.0 meter. It was a realization that the concepts we have come to “associate” with the Web 2.0 were really the Web 1.0’s original goals. GAH!!! I just had an unpleasant realization that now I am trying to quantify the Web 2.0. To solve this problem, I think we have to look to the past as well as the future (yeah yeah – so what does that rule out? Thinking about the exact present moment?). The Oreilly folks are pretty smart IMO so please don't take this as some petty stab - more of a friendly prod :-)
If “Web 2.0” is to be used as a catch all term for where we are going, let's put some substance behind it. If not, it will suffer the same symptoms as SOA and Web Services - both very meaningful to most people, just with differing semantics. In fact, the lack of clarity around SOA lead a group of almost 200 people to get together and write a formal Reference Model for SOA under the auspices of OASIS. Similarly, a group got together within W3C and worked on a Reference Architecture for Web Services. I am proud to state that I worked on both projects.
So what can be done to put substance in the Web 2.0?
1. Write an abstract reference model to show the components of the Web 2.0 (abstract); and
2. Create sets of high level abstract patterns, mid level patterns and low level idioms to illustrate what is really meant by the web 2.0; and
3. Create reference architecture (in plural and somewhat generic) for all components of the Web 2.0, describing their externally visible properties and relationships with other components.
An example would be to illustrate the syndication-subscription pattern using some architectural patterns template. The abstract notion is that subscribers notify a syndication component of their wish to receive content. When the syndication component has content it is ready to push out, it configures a list of recipients based on some criteria then proceeds to push the content out. A lower level idiom could show this implemented using Apache components, perhaps even with options for content formatting based on device, reliable messaging protocols, security and end user authentication with a persistent security model for the content itself.
The cool thing about this approach is that it still gives each and every implementer the freedom to make their own black box components whilst preserving a common layer of understanding. It also provides documentation about what is really meant, granted, those who cannot distinguish abstract from concrete may still be confused.
Of course to do this, you would require an architectural patterns meta model and template that allowed you to go from the very abstract to the very concrete, but I think I know where one is that can be donated to some organization.
Why should this be done? Simple – without this, “Web 2.0” is nothing more that a marketing term. Sure – several people will say “no – it means X and exactly X”, but the chances of Boolean Y = eval(personA.X == personB.X) evaluating to “1” in every instance is very low IMHO.
The Web 1.0, aka the “internet”, has achieved a common definition though. Even though it is not concisely written, there is general consensus on what a web server does, what the layered wire protocols do, how security works and how people interact with websites (via browsers). If someone says “this server is internet enabled”, people imply that it means it can take HTTP requests and return text in compliance with the requesters requirements.
Sorry – folks. I just don’t believe that the Web 2.0 will inherit an implied reference model the way the Web 1.0 did. The culprit for this is the Web 1.0 as it exists – it allows anyone, almost anywhere, to write what they think the Web 2.0 is and share it with others. Also, unlike the Web 1.0, the Web 2.0 is not mandatory. The basic components of the Web 1.0 such as HTTP, TCP/IP, SMTP, MIME, HTML etc all were mandatory, therefore it was fairly easy to draw a box and state – this is the Web 1.0. One could even through in some common non-mandatory components and still make a solid statement (example – scripting languages (ASP, VBScript, JavaScript, ActionScript plus CSS, XML et al).
A Reference Model for the Web 2.0 might want to declare some form of compliancy and conformancy statements. Such might be a weighted test of it could be a bar that you must pass. Regardless, before this exists, what is the point of building “validators” and “meters”. Harrumph – end of rant. Take it all with a grain of salt – the Oreilly folks are smart and I’m sure we’ll see something soon ;-)
If not – does anyone feel compelled to take a stab at a formal definition? I will gladly jump in the fray and donate my time to help.
*Yet Another Blog Article – in case you didn’t figure it out ;-)
Friday, March 24, 2006
Flex 2.0 beta 2 thoughts
Flex has been getting a lot more press lately as we continue to release betas building up to 2.0. One thing is clear from my perspective - about 98% of all people who are exposed to it are both blown away and excited by it. IMO - this is significant.
One of the strongest proponents of Flex told me that this is due to the speed in which non-flash designers can build a flash based GUI for the web. Flash has typically been a tool used by the creative professional side of the equation to deliver a rich user experience. Flash has not been a skill set owned by developers but developers have always required a way to quickly codify visual interfaces for the products of their domain.
Typically, Eclipse based IDE's have granted developers the ease for building visual front ends to their products. Most of these have been java based derivatives to make Swing, JSP or other GUI's. The pervasive issue with all of these is consistent rendering. HTML heads can tell you that the prospect of a webpage rendering consistently on multiple browsers exponentially falls off as you incorporate more dynamic aspects of HTML 4.0 such as layers, div's and javascripting.
Flex and Flash are a match made in heaven. Because the final product is Flash, it renders consistently given the free Flash player isolates it from operating system and browser dependencies.
If you haven’t installed the latest beta 2, you can download it from here. You will need to read the installation instructions however. All previous betas have to be completely removed.
One of the strongest proponents of Flex told me that this is due to the speed in which non-flash designers can build a flash based GUI for the web. Flash has typically been a tool used by the creative professional side of the equation to deliver a rich user experience. Flash has not been a skill set owned by developers but developers have always required a way to quickly codify visual interfaces for the products of their domain.
Typically, Eclipse based IDE's have granted developers the ease for building visual front ends to their products. Most of these have been java based derivatives to make Swing, JSP or other GUI's. The pervasive issue with all of these is consistent rendering. HTML heads can tell you that the prospect of a webpage rendering consistently on multiple browsers exponentially falls off as you incorporate more dynamic aspects of HTML 4.0 such as layers, div's and javascripting.
Flex and Flash are a match made in heaven. Because the final product is Flash, it renders consistently given the free Flash player isolates it from operating system and browser dependencies.
If you haven’t installed the latest beta 2, you can download it from here. You will need to read the installation instructions however. All previous betas have to be completely removed.
Thursday, March 23, 2006
My comments to the WTO and the Chinese Ministry of Commerce on IPR issues
The Chinese Ministry of Commerce recently asked for a response back to a paper entitled "Background paper for Chinese Submission to WTO on Intellectual Property Right Issues in Standardization". I feel very strongly about this issue. The red text below is my response (mine alone and in no way reflects that of my employer or anybody else). As Joey Sh!thead Keithley from DOA once said "Talk - Action = Zero". J - hope this fits the attitude!! TEXT BELOW:
Draft Background paper for Chinese Submission to WTO on Intellectual Property Right Issues in Standardization
(G/TBT/W/251)
I. Incorporation of Intellectual Property Rights into Technical Standards is an Inevitable Outcome of the Development of Science, Technology and Economy.
Standards reflect technology evolution. Incorporating fruits of science advance and technology innovation in standards is one distinctive feature and a trend of contemporary standardization effort worldwide. Intellectual property rights(IPRs), especially patent, are technical solutions that embody such science advances and technology innovations. Consequently, the technical solutions incorporated in standards may well coincide with those in patented technologies.
2.Nowadays, it is common practices to protect corporate R&D results through IPR regime. Patented technologies almost reach every corner of all sectors, especially in high-tech ones such as information and telecommunication (ICT), where virtually all the key technologies of market significance are loaded with IPRs.
3.Implementation of all kinds of standards, be them descriptive/design or functional, may involve use of patented technologies. Even if a standard only defines the performance of a product rather than specific technical features, the implementation of the standard may still implicate IPRs, since the technical solutions to achieve the performance target can be covered by IPRs, i.e., IPRs can be indirectly included into standards that set requirements for product performance.
4.Because of the great difficulties and huge cost in retrieving, analyzing and comparing technical solution to determine its proprietary state, it’s impractical to avoid IPR in standard-setting.
5. Enterprises holding IPRs, especially essential IPRs, make significant input in worldwide standard setting activities. This is particularly true in high-tech sectors, where standardizations are often initiated and mainly advanced by those enterprises equipped with key and sophisticated proprietary technologies.
II.Issues Regarding Combination of IPR and Technical Standards and its Impact on International Trade.
6. WTO’s World Trade Report 2005 – Exploring the links between trade, standards and the WTO cited a research which “found a positive correlation between patent applications and new technical regulations, especially in innovative fields”. Based on a cross-country analysis, this research also found “that sectors with a higher propensity for standardization tend to be more patent-intensive and export intensive”. It is easy to foresee that there is an obvious trend of proprietary technology entering into standards.
7. Standard Development Organizations (SDO), at international, regional and national levels, including ISO, IEC, ITU, ETSI, ANSI, etc, have neither avoided, nor can they avoid, the inclusion of patented technologies. Therefore, on the issue of incorporation patented technologies into standards, they in principle have no objection against patented items. Duane: In principle, this assumption may be flawed. Some SDO’s do have IPR policies which address this. Each one has their own IPR policy and they are not consistent, as noted later in this paper. It is also notable that it is not always possible for the SDO to have objections or acquiesce if they are not aware of the patents. Sometimes, a company may not make a patent available to all employees and it is possible that one employee may participate in a standards development process and create something unaware that someone else in his own company invented the same thing independently earlier on. Many Patent processes do address this but they are usually typical to one region and not addressed to work with large Multi-national Corporations (MNC’s) that may have subsidiaries in different countries. IMO – a common set of guidelines might be a good idea that are applied consistently across all SDO’s. I am curious about how you see this this work developing. It is a worthy cause.
8. In 2005, List of IEC patent declarations received by IEC posted 27 patent-related information releases, where one release may cover a lot of patents. For instance, in the information release dated Nov. 14, 2005, Zebra Technology Company claimed 98 patents worldwide on ISO/IEC 18000-6 standard. By February 8th, 2006, ITU-T Patent Statement and Licensing Declaration Database contained 1494 patent declarations, among them 137 were received after the approvals of the Recommendations. Up to 1998, ETSI posted 72 patent releases in 16 standardization sectors. By the late 1990s, over 20 companies claimed to hold about 140 patents which they construed as ‘essential' to the GSM standard.
DN: Would you be so kind as to back this up with references? Several of the groups mentioned have alleged this may be inaccurate. If you could please provide specific references?
9. Combination of IPRs with standards may be problematic and bring about negative impact on standardization and international trade.
Duane: It may be so, however there are IPR policies in place that do protect those who work with the standards subsequently. It is dependent upon many factors including the inclusion of RAND or donation of IPR to the SDO. The industry is very self governing given the very existence of software companies is dependent upon understanding these issues.
10. With regard to IPR declaration, patent holders may hold back patent information in the process of standard setting in the hope for unfair interests. In this regard, Dell case is illustrative.
12. When IPR holders refuse to license their proprietary technologies on RAND terms, standardizing efforts will confront crises. In accordance with the prevailing patent policies of SDOs, if the identified patent holders refuse to license on RAND terms and conditions, the SDOs can alter the standard around the proprietary technology. Yet it should be noted that some essential technologies are hard to avoid. If that is the case, the standard at issue may have to be withdrawn. Standard setting works have suffered, are suffering and will continue to suffer such inefficiency. As a commentator put it, “One of the most difficult areas in standards development these days is that of intellectual property rights (IPR). Internet Engineering Task Force (IETF) has seen many cases of submarine patents and other attempts to subvert the standards process. This is not just a problem for the IETF; many other standards development organizations have also had problems in this area.”
DN: As per my comments on the previous pages, this may not always be intentional. The end result may be a combination of both internal process and size of organization.
13. In light of foregoing facts, inclusion IPR into standards may have serious impact on the international standards setting efforts and the corresponding implementations. As TBT Agreement aims at boosting production efficiency and facilitating international trade by encouraging the adoption of international standards, such objectives can be frustrated and therefore international trade retarded. According to WTO’s World Trade Report 2005, the patent dispute on 3G standard retarded the commercialization of 3G mobile phone, which only just started at the end of 2004 while ITU approved the International Mobile Telecommunication 2000(IMT-2000) in 2000. 3G standard patent dispute litigations are still active and in high profile. Disputes of this kind will cause negative impact on 3G standard implementation, commercialization and related international trade.
Duane:Possibly true, however common sense seems to dictate that it is better to have a widely adopted standard in place to mitigate these types of things given it creates a large base of common stakeholders who use the standard, collectively having more bargaining power than individuals. If the allegations as suggested are true and worrisome, I would rather be part of a large coalition with a common interest and a published IPR policy to work with rather than using a proprietary technology and being by myself.
III. The International Communities are Paying Much Attention to IPR Issues in Standardization.
14. The impacts of IPRs on economic and trade development and the urgency to address them had been widely recognized. Related actions have been taken or are being taken.
15. The international standardization setting bodies like ISO ÿ IEC and ITU-T have recognized the impact of IPR issues on standard setting and implementation., and have made great endeavor to solve problems related to IPR issues in standardization. They had formulated basic principles for patent disclosure and licensing arrangements, which are widely cited by other SDOs. These principles also constitute sound technical base and a roadmap for the discussion in WTO. It should also be highlighted that ISO, IEC and ITU-T have taken into consideration the development needs in their policies and activities. For example, ISO has established a specific committee (ISO/DEVCO) looking after developments dimension of international standardization activities. ITU-T has several Resolutions in place to bridge the standardization gap between developing and developed members.
16. Aside from the patent policies of international SDOs, United Nation Center for Trade Facilitation and Electronic Business (UNCEFACT) is also well aware of this issue in its standardization work and begins to formulate its patent policy for standards on electronic commerce. In some developed countries, these issues have captured the regulators’ attention.
DN: I agree. It is imperative that UN/CEFACT complete and publish its final IPR policy as soon as possible. This cannot wait any longer.
In Japan, the Guidelines for Patent and Know-how Licensing Agreements under the Antimonopoly Act implemented by Japan Fair Trade Commission, stipulated that the patent holders whose patents implicated by standards adopted by government agencies shall not use their patents to exclude or control other companies, including exclusion and/or control of commercial activities of the patent users. In U.S., the Federal Trade Commission (FTC) and Department of Justice (DOJ) had been conducting intense discussions on these issues, including the 2002 FTC/DOJ joint Hearings on Competition and Intellectual Property Law and Policy in the Knowledge-Based Economy. In EU, EC Communication on IPRs and Standardization recognized that IPR holders should “make best efforts to identify any IPR which they hold relevant to a standard under development and to confirm or refuse permission for its incorporation in the standard”; “Offer fair, reasonable and non-discriminatory monetary or non-monetary terms for the license to use IPR” ; and “Treat their eventual agreement for incorporating an IPR in a standard as irrevocable.” In September 2005, Mr. Ryutaro Matsumoto, Japanese Industrial Standards Committee (JISC) Secretary General and Director General of METI sent a letter to ISO and IEC urging ISO and IEC to establish the IPR as hoc group so that member countries can participate in the discussions on the IPR issues and to clarify the operation of their patent policies.
Duane: The expressed urgency coming from a member delegation from a government who has notified the World Intellectual Property Organization (WIPO) that certain special economic regions like Macau are not going to honor conditions of the Madrid protocol seems a bit hypocritical to me:
July 7, 2000:
“The Permanent Mission of the People's Republic of China to the United Nations Office in Geneva and other International Organizations in Switzerland presents its compliments to Dr. Kamil Idris, Director General of the World Intellectual Property Organization, and has the honor to inform the latter that, till further notice from the Chinese Government, the Patent Cooperation Treaty (PCT), the Madrid Agreement Concerning the International Registration of Marks, to which the People's Republic of China is a member State, are not applicable to the Macao Special Administrative Region of China.”
http://www.wipo.int/edocs/notdocs/en/madrid-gp/treaty_madrid_gp_126.html
IV. To Strike Balance between IPR Holders and Standard Implementers for a
Win-Win Situation.
17. The discussion on IPR issues in Standardization does not mean that IPR holders will lose and the IPR users will gain. The real problem stands now is that there are no sufficient rules to respond to IPR issues in standardization within international community, including WTO framework. Without well-defined rule to follow, inefficiency arises and disputes result to the detriments of both IPR holders and IPR users, who come from both developing and developed Members. While it is important to protect the rights and interests of IPR holders, it’s equally significant that new international standards and advanced IPR technologies are applied as widely as possible in order to enhance efficient, high quality production and to facilitate world trade to the interests of consumers worldwide. If there is a rule of balance that sufficiently clear to follow, the application of new international standard and new advanced IPR technologies will be smooth, and benefits will accrue to IPR holders--more users, more profit.
DN: I would not agree with all these assertions. There are certainly documented processes for dealing with IPR issues pertaining to standards development. They are readily available from several websites of SDO’s. IMO – these are well defined and easy to administrate and have worked for some time in general. If there are specific things in these published processes you have issues with, the correct path might be to notify the correct SDO to illustrate the problem. Making wide sweeping remarks contrary to the status quo will not likely resolve concerns. I would not posture that all is perfect and I believe the best thing to move forward would be to have the concerned parties come forward with concrete suggestions to address the issues rather than vague claims it doesn’t work. In lack of a concrete solution, things may just remain as they are.
V. Legitimacy to Discuss IPR Issues in Standardization within WTO TBT
Committee.
18. Article 2.4 of the TBT Agreement encourages WTO Members to adopt international standards as the basis of the standards and technical regulations. If Members are not clear of IPRs in the relevant international standard, whether all the IPRs have been disclosed, under what terms the IPRs are to be licensed by the IPR holders, all WTO Members will face difficulties when adopting international standards. In the case that the national technical regulations are adopted, the enterprises who are subject to implementation of the technical regulations will encounter great difficulties with relation to disclosure of IPRs in standards and hard and time-consuming negotiations with IPR holders on the terms of licensing. Either the IPR holders are domestic or foreign companies, every Member will have to face this problem, which actually is the problem between the IPR holders and IPR users. Standards developed by credible SDO’s with documented IPR policies should be easier to digest for end users.
Duane: You are also leaving out another piece of the puzzle which is the ISV (Independent Software vendor). ISV’s also have to concern themselves with IPR when they build products and solutions that implement the standards, as do open source foundations. End users can isolate and mitigate many of these problems by purchasing software from credible vendors or using open source software from organizations who have already done their IPR homework and will back it in their EULA. I think that the WTO should not tread on this area and simply leave it to the reputable standards bodies to create and manage standards with acceptable IPR policies. The WTO can assess the SDO products to ensure they meet the needs of the end users and make recommendations based on their findings.
19. From the governmental level, as well as the company level, there exists the kind of unwillingness of adopting international standards as the basis of their national standards and technical regulations if there is no common rule to regulate IPRs in standardization. Such a situation will bring negative impact on implementation of TBT Agreement with relation to adoption of international standards while this Committee encourages Members to adopt international standards to their maximum extent. With a view to facilitating the setting and implementation of international standards, therefore the smooth implementation of the TBT Agreement, IPR issues in standardization must be addressed properly.
Duane: There is a group called WIPO which China is a member of. I believe their work, including the PCT, Madrid Protocol, Paris convention and other works lays the foundational framework for global IPR issues. The mechanism itself belongs in the hands of WIPO and each SDO should strive to ensure full disclosure to the end users. Left the users decide.
20. China proposes that international standard setting bodies, as well as Members, provide the Committee with relevant information regarding practices and experience on their IPR policies in standardization for Members’ understanding and reference. Information exchange will be helpful and necessary to facilitate meaningful discussions.
Duane: I would concur with this point. It should be noted that many SDO’s have their IPR policies published or referenced at the bottom of each specification. The ones who do not need to ensure they can for the future.
21. The discussion on IPR issues in standardization is of great significance to the integrity of international standardization community and multilateral trade system. China is of the belief that this issue should be carried forward within WTO. China will continue to make her constructive contributions in this regard.
Duane: I would state that WIPO and SDO’s are the place where this belongs. They can collectively make recommendations for WTO to support the articles of the DOHA accord that require technical standards to enable. Along with a fully disclosed IPR policy, the WTO should provide guidelines to the SDO’s such as UN/CEFACT that they wish to have in the final products and let the technical SDO’s perform the tactical work to support the WTO.
Electronic version of this Background paper is available at request.
Comments and suggestions please be sent to:
Mr. AN Baisheng
Deputy Division Director
Department for WTO Affairs, MOFCOM, PRC
Tel: 8610 65197242, Fax: 8610 6512 8304
Email: anbaisheng@mofcom.gov.cn
Ms. GUO Xueyan
First Secretary,
Permanent Mission to WTO of the People’s Republic of China
TeL: 4122 9097685, Fax: 4122 9097699
Email:guoxueyan9999@gmail.com
Draft Background paper for Chinese Submission to WTO on Intellectual Property Right Issues in Standardization
(G/TBT/W/251)
I. Incorporation of Intellectual Property Rights into Technical Standards is an Inevitable Outcome of the Development of Science, Technology and Economy.
Standards reflect technology evolution. Incorporating fruits of science advance and technology innovation in standards is one distinctive feature and a trend of contemporary standardization effort worldwide. Intellectual property rights(IPRs), especially patent, are technical solutions that embody such science advances and technology innovations. Consequently, the technical solutions incorporated in standards may well coincide with those in patented technologies.
2.Nowadays, it is common practices to protect corporate R&D results through IPR regime. Patented technologies almost reach every corner of all sectors, especially in high-tech ones such as information and telecommunication (ICT), where virtually all the key technologies of market significance are loaded with IPRs.
3.Implementation of all kinds of standards, be them descriptive/design or functional, may involve use of patented technologies. Even if a standard only defines the performance of a product rather than specific technical features, the implementation of the standard may still implicate IPRs, since the technical solutions to achieve the performance target can be covered by IPRs, i.e., IPRs can be indirectly included into standards that set requirements for product performance.
4.Because of the great difficulties and huge cost in retrieving, analyzing and comparing technical solution to determine its proprietary state, it’s impractical to avoid IPR in standard-setting.
5. Enterprises holding IPRs, especially essential IPRs, make significant input in worldwide standard setting activities. This is particularly true in high-tech sectors, where standardizations are often initiated and mainly advanced by those enterprises equipped with key and sophisticated proprietary technologies.
II.Issues Regarding Combination of IPR and Technical Standards and its Impact on International Trade.
6. WTO’s World Trade Report 2005 – Exploring the links between trade, standards and the WTO cited a research which “found a positive correlation between patent applications and new technical regulations, especially in innovative fields”. Based on a cross-country analysis, this research also found “that sectors with a higher propensity for standardization tend to be more patent-intensive and export intensive”. It is easy to foresee that there is an obvious trend of proprietary technology entering into standards.
7. Standard Development Organizations (SDO), at international, regional and national levels, including ISO, IEC, ITU, ETSI, ANSI, etc, have neither avoided, nor can they avoid, the inclusion of patented technologies. Therefore, on the issue of incorporation patented technologies into standards, they in principle have no objection against patented items. Duane: In principle, this assumption may be flawed. Some SDO’s do have IPR policies which address this. Each one has their own IPR policy and they are not consistent, as noted later in this paper. It is also notable that it is not always possible for the SDO to have objections or acquiesce if they are not aware of the patents. Sometimes, a company may not make a patent available to all employees and it is possible that one employee may participate in a standards development process and create something unaware that someone else in his own company invented the same thing independently earlier on. Many Patent processes do address this but they are usually typical to one region and not addressed to work with large Multi-national Corporations (MNC’s) that may have subsidiaries in different countries. IMO – a common set of guidelines might be a good idea that are applied consistently across all SDO’s. I am curious about how you see this this work developing. It is a worthy cause.
8. In 2005, List of IEC patent declarations received by IEC posted 27 patent-related information releases, where one release may cover a lot of patents. For instance, in the information release dated Nov. 14, 2005, Zebra Technology Company claimed 98 patents worldwide on ISO/IEC 18000-6 standard. By February 8th, 2006, ITU-T Patent Statement and Licensing Declaration Database contained 1494 patent declarations, among them 137 were received after the approvals of the Recommendations. Up to 1998, ETSI posted 72 patent releases in 16 standardization sectors. By the late 1990s, over 20 companies claimed to hold about 140 patents which they construed as ‘essential' to the GSM standard.
DN: Would you be so kind as to back this up with references? Several of the groups mentioned have alleged this may be inaccurate. If you could please provide specific references?
9. Combination of IPRs with standards may be problematic and bring about negative impact on standardization and international trade.
Duane: It may be so, however there are IPR policies in place that do protect those who work with the standards subsequently. It is dependent upon many factors including the inclusion of RAND or donation of IPR to the SDO. The industry is very self governing given the very existence of software companies is dependent upon understanding these issues.
10. With regard to IPR declaration, patent holders may hold back patent information in the process of standard setting in the hope for unfair interests. In this regard, Dell case is illustrative.
12. When IPR holders refuse to license their proprietary technologies on RAND terms, standardizing efforts will confront crises. In accordance with the prevailing patent policies of SDOs, if the identified patent holders refuse to license on RAND terms and conditions, the SDOs can alter the standard around the proprietary technology. Yet it should be noted that some essential technologies are hard to avoid. If that is the case, the standard at issue may have to be withdrawn. Standard setting works have suffered, are suffering and will continue to suffer such inefficiency. As a commentator put it, “One of the most difficult areas in standards development these days is that of intellectual property rights (IPR). Internet Engineering Task Force (IETF) has seen many cases of submarine patents and other attempts to subvert the standards process. This is not just a problem for the IETF; many other standards development organizations have also had problems in this area.”
DN: As per my comments on the previous pages, this may not always be intentional. The end result may be a combination of both internal process and size of organization.
13. In light of foregoing facts, inclusion IPR into standards may have serious impact on the international standards setting efforts and the corresponding implementations. As TBT Agreement aims at boosting production efficiency and facilitating international trade by encouraging the adoption of international standards, such objectives can be frustrated and therefore international trade retarded. According to WTO’s World Trade Report 2005, the patent dispute on 3G standard retarded the commercialization of 3G mobile phone, which only just started at the end of 2004 while ITU approved the International Mobile Telecommunication 2000(IMT-2000) in 2000. 3G standard patent dispute litigations are still active and in high profile. Disputes of this kind will cause negative impact on 3G standard implementation, commercialization and related international trade.
Duane:Possibly true, however common sense seems to dictate that it is better to have a widely adopted standard in place to mitigate these types of things given it creates a large base of common stakeholders who use the standard, collectively having more bargaining power than individuals. If the allegations as suggested are true and worrisome, I would rather be part of a large coalition with a common interest and a published IPR policy to work with rather than using a proprietary technology and being by myself.
III. The International Communities are Paying Much Attention to IPR Issues in Standardization.
14. The impacts of IPRs on economic and trade development and the urgency to address them had been widely recognized. Related actions have been taken or are being taken.
15. The international standardization setting bodies like ISO ÿ IEC and ITU-T have recognized the impact of IPR issues on standard setting and implementation., and have made great endeavor to solve problems related to IPR issues in standardization. They had formulated basic principles for patent disclosure and licensing arrangements, which are widely cited by other SDOs. These principles also constitute sound technical base and a roadmap for the discussion in WTO. It should also be highlighted that ISO, IEC and ITU-T have taken into consideration the development needs in their policies and activities. For example, ISO has established a specific committee (ISO/DEVCO) looking after developments dimension of international standardization activities. ITU-T has several Resolutions in place to bridge the standardization gap between developing and developed members.
16. Aside from the patent policies of international SDOs, United Nation Center for Trade Facilitation and Electronic Business (UNCEFACT) is also well aware of this issue in its standardization work and begins to formulate its patent policy for standards on electronic commerce. In some developed countries, these issues have captured the regulators’ attention.
DN: I agree. It is imperative that UN/CEFACT complete and publish its final IPR policy as soon as possible. This cannot wait any longer.
In Japan, the Guidelines for Patent and Know-how Licensing Agreements under the Antimonopoly Act implemented by Japan Fair Trade Commission, stipulated that the patent holders whose patents implicated by standards adopted by government agencies shall not use their patents to exclude or control other companies, including exclusion and/or control of commercial activities of the patent users. In U.S., the Federal Trade Commission (FTC) and Department of Justice (DOJ) had been conducting intense discussions on these issues, including the 2002 FTC/DOJ joint Hearings on Competition and Intellectual Property Law and Policy in the Knowledge-Based Economy. In EU, EC Communication on IPRs and Standardization recognized that IPR holders should “make best efforts to identify any IPR which they hold relevant to a standard under development and to confirm or refuse permission for its incorporation in the standard”; “Offer fair, reasonable and non-discriminatory monetary or non-monetary terms for the license to use IPR” ; and “Treat their eventual agreement for incorporating an IPR in a standard as irrevocable.” In September 2005, Mr. Ryutaro Matsumoto, Japanese Industrial Standards Committee (JISC) Secretary General and Director General of METI sent a letter to ISO and IEC urging ISO and IEC to establish the IPR as hoc group so that member countries can participate in the discussions on the IPR issues and to clarify the operation of their patent policies.
Duane: The expressed urgency coming from a member delegation from a government who has notified the World Intellectual Property Organization (WIPO) that certain special economic regions like Macau are not going to honor conditions of the Madrid protocol seems a bit hypocritical to me:
July 7, 2000:
“The Permanent Mission of the People's Republic of China to the United Nations Office in Geneva and other International Organizations in Switzerland presents its compliments to Dr. Kamil Idris, Director General of the World Intellectual Property Organization, and has the honor to inform the latter that, till further notice from the Chinese Government, the Patent Cooperation Treaty (PCT), the Madrid Agreement Concerning the International Registration of Marks, to which the People's Republic of China is a member State, are not applicable to the Macao Special Administrative Region of China.”
http://www.wipo.int/edocs/notdocs/en/madrid-gp/treaty_madrid_gp_126.html
IV. To Strike Balance between IPR Holders and Standard Implementers for a
Win-Win Situation.
17. The discussion on IPR issues in Standardization does not mean that IPR holders will lose and the IPR users will gain. The real problem stands now is that there are no sufficient rules to respond to IPR issues in standardization within international community, including WTO framework. Without well-defined rule to follow, inefficiency arises and disputes result to the detriments of both IPR holders and IPR users, who come from both developing and developed Members. While it is important to protect the rights and interests of IPR holders, it’s equally significant that new international standards and advanced IPR technologies are applied as widely as possible in order to enhance efficient, high quality production and to facilitate world trade to the interests of consumers worldwide. If there is a rule of balance that sufficiently clear to follow, the application of new international standard and new advanced IPR technologies will be smooth, and benefits will accrue to IPR holders--more users, more profit.
DN: I would not agree with all these assertions. There are certainly documented processes for dealing with IPR issues pertaining to standards development. They are readily available from several websites of SDO’s. IMO – these are well defined and easy to administrate and have worked for some time in general. If there are specific things in these published processes you have issues with, the correct path might be to notify the correct SDO to illustrate the problem. Making wide sweeping remarks contrary to the status quo will not likely resolve concerns. I would not posture that all is perfect and I believe the best thing to move forward would be to have the concerned parties come forward with concrete suggestions to address the issues rather than vague claims it doesn’t work. In lack of a concrete solution, things may just remain as they are.
V. Legitimacy to Discuss IPR Issues in Standardization within WTO TBT
Committee.
18. Article 2.4 of the TBT Agreement encourages WTO Members to adopt international standards as the basis of the standards and technical regulations. If Members are not clear of IPRs in the relevant international standard, whether all the IPRs have been disclosed, under what terms the IPRs are to be licensed by the IPR holders, all WTO Members will face difficulties when adopting international standards. In the case that the national technical regulations are adopted, the enterprises who are subject to implementation of the technical regulations will encounter great difficulties with relation to disclosure of IPRs in standards and hard and time-consuming negotiations with IPR holders on the terms of licensing. Either the IPR holders are domestic or foreign companies, every Member will have to face this problem, which actually is the problem between the IPR holders and IPR users. Standards developed by credible SDO’s with documented IPR policies should be easier to digest for end users.
Duane: You are also leaving out another piece of the puzzle which is the ISV (Independent Software vendor). ISV’s also have to concern themselves with IPR when they build products and solutions that implement the standards, as do open source foundations. End users can isolate and mitigate many of these problems by purchasing software from credible vendors or using open source software from organizations who have already done their IPR homework and will back it in their EULA. I think that the WTO should not tread on this area and simply leave it to the reputable standards bodies to create and manage standards with acceptable IPR policies. The WTO can assess the SDO products to ensure they meet the needs of the end users and make recommendations based on their findings.
19. From the governmental level, as well as the company level, there exists the kind of unwillingness of adopting international standards as the basis of their national standards and technical regulations if there is no common rule to regulate IPRs in standardization. Such a situation will bring negative impact on implementation of TBT Agreement with relation to adoption of international standards while this Committee encourages Members to adopt international standards to their maximum extent. With a view to facilitating the setting and implementation of international standards, therefore the smooth implementation of the TBT Agreement, IPR issues in standardization must be addressed properly.
Duane: There is a group called WIPO which China is a member of. I believe their work, including the PCT, Madrid Protocol, Paris convention and other works lays the foundational framework for global IPR issues. The mechanism itself belongs in the hands of WIPO and each SDO should strive to ensure full disclosure to the end users. Left the users decide.
20. China proposes that international standard setting bodies, as well as Members, provide the Committee with relevant information regarding practices and experience on their IPR policies in standardization for Members’ understanding and reference. Information exchange will be helpful and necessary to facilitate meaningful discussions.
Duane: I would concur with this point. It should be noted that many SDO’s have their IPR policies published or referenced at the bottom of each specification. The ones who do not need to ensure they can for the future.
21. The discussion on IPR issues in standardization is of great significance to the integrity of international standardization community and multilateral trade system. China is of the belief that this issue should be carried forward within WTO. China will continue to make her constructive contributions in this regard.
Duane: I would state that WIPO and SDO’s are the place where this belongs. They can collectively make recommendations for WTO to support the articles of the DOHA accord that require technical standards to enable. Along with a fully disclosed IPR policy, the WTO should provide guidelines to the SDO’s such as UN/CEFACT that they wish to have in the final products and let the technical SDO’s perform the tactical work to support the WTO.
Electronic version of this Background paper is available at request.
Comments and suggestions please be sent to:
Mr. AN Baisheng
Deputy Division Director
Department for WTO Affairs, MOFCOM, PRC
Tel: 8610 65197242, Fax: 8610 6512 8304
Email: anbaisheng@mofcom.gov.cn
Ms. GUO Xueyan
First Secretary,
Permanent Mission to WTO of the People’s Republic of China
TeL: 4122 9097685, Fax: 4122 9097699
Email:guoxueyan9999@gmail.com
Tuesday, February 21, 2006
Reference Model for SOA out for review
After an exhausting year, the OASIS SOA RM Technical Committee has finally turned out a stable draft defining a reference model for service oriented architecture. The specification is now out for public review and available from the TC's main web pages at:
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=soa-rm
The 200+ people who worked on this deserve a lot of credit for arguably creating a work which should withstand the current SOA hype. Consensus was difficult but we got there. The team is great.
The Reference Model is abstract in nature. If you read it and expect to see things like a "Service Consumer" and "Service Provider" you will be sadly disappointed. Those are concrete things and belong in architecture, not an abstract model. Those of you with architectural backgrounds will probably find this very useful however and understand the value.
I would assert that the model itself is probably what the W3C Web Services Architecture Group should have delivered rather than attempting to define a grandiose architecture for web services. The one size fits all attitude WRT SOA simply does not scale to all, given the vast variations in requirements.
So what good is this reference model? Simply - it is a stick in the mud (perhaps FUD). If you agree with it, you can state "when I say SOA, I mean SOA as spelled out by the OASIS RM for SOA". If you disagree with it, you can say "when I say SOA, I mean SOA spelled out by the OASIS RM for SOA, with the following exceptions....[place_your_objections_here]". Not that definitions of SOA are rare (in fact there are probably about 6.5 billion on the planet), but honestly, the entire industry benefits from a standard definition that no one group owns.
It also is a guide to be used by architects who are embarking on SOA projects. If everyone in the industry was also to have a common understanding of SOA, it would probably make all our jobs a lot easier.
I heard a really funny statement from an analyst the other day too. They stated quite emphatically that "65% of companies are not doing SOA now, but expect to be by 2007". WTF??? IMHO - if you are reading this blog entry now, you have used the patterns of SOA as defined by the RM. If you send email you are too. I guess "Doing SOA" sounds more risque.
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=soa-rm
The 200+ people who worked on this deserve a lot of credit for arguably creating a work which should withstand the current SOA hype. Consensus was difficult but we got there. The team is great.
The Reference Model is abstract in nature. If you read it and expect to see things like a "Service Consumer" and "Service Provider" you will be sadly disappointed. Those are concrete things and belong in architecture, not an abstract model. Those of you with architectural backgrounds will probably find this very useful however and understand the value.
I would assert that the model itself is probably what the W3C Web Services Architecture Group should have delivered rather than attempting to define a grandiose architecture for web services. The one size fits all attitude WRT SOA simply does not scale to all, given the vast variations in requirements.
So what good is this reference model? Simply - it is a stick in the mud (perhaps FUD). If you agree with it, you can state "when I say SOA, I mean SOA as spelled out by the OASIS RM for SOA". If you disagree with it, you can say "when I say SOA, I mean SOA spelled out by the OASIS RM for SOA, with the following exceptions....[place_your_objections_here]". Not that definitions of SOA are rare (in fact there are probably about 6.5 billion on the planet), but honestly, the entire industry benefits from a standard definition that no one group owns.
It also is a guide to be used by architects who are embarking on SOA projects. If everyone in the industry was also to have a common understanding of SOA, it would probably make all our jobs a lot easier.
I heard a really funny statement from an analyst the other day too. They stated quite emphatically that "65% of companies are not doing SOA now, but expect to be by 2007". WTF??? IMHO - if you are reading this blog entry now, you have used the patterns of SOA as defined by the RM. If you send email you are too. I guess "Doing SOA" sounds more risque.
Thursday, February 09, 2006
Flex is going to be HUGE!!
Okay - I have had a couple weeks now to play around with the new Flex builder 2.0 beta. One word - amazing! I rarely get excited about technology these days given most "new" things are simply a minor gain on something already done, but quote me on this - Flex is going to be huge. It's not that Flex is obscure now, it is just that once developers and other tech-heads use it, they will probably largely abandon what they are using now for front ends.
During Adobe's technology Summit, I had the opportunity to watch some of my colleagues build a Flickr Photo browser from scratch, using the web API, in about 10 minutes. While I was watching this, my colleague Matt Mackenzie was sitting beside me and not only duplicated it, but was busy making mods. This is stuff that used to take days/weeks using JSP's! The processing model is also perfect given it removes variances for different platforms. No longer will this be an issue as long as the user is using the reference implementation of Flash Player.
Developers - if you haven't used it yet, download it now from here:
http://labs.macromedia.com/
Analysts, Press, Media types - this is going to be big. Learn about it now from the same URL.
During Adobe's technology Summit, I had the opportunity to watch some of my colleagues build a Flickr Photo browser from scratch, using the web API, in about 10 minutes. While I was watching this, my colleague Matt Mackenzie was sitting beside me and not only duplicated it, but was busy making mods. This is stuff that used to take days/weeks using JSP's! The processing model is also perfect given it removes variances for different platforms. No longer will this be an issue as long as the user is using the reference implementation of Flash Player.
Developers - if you haven't used it yet, download it now from here:
http://labs.macromedia.com/
Analysts, Press, Media types - this is going to be big. Learn about it now from the same URL.
Wednesday, February 01, 2006
Flex 2.0 Beta 1 is available now!!!
Okay - no blog entry for a while so I'll make this one worth a read. Yes - Beta 2.0 is out now. There is a download available which includes Flex Framework 2.0, Flex Builder 2.0, Flex Enterprise Services 2.0, Flex Charting Components 2.0, ActionScript 3.0, Flash Player 8.5.
http://labs.macromedia.com/
Flex Enterprise Services 2.0 enables a more productive way to build data-intensive Flex applications. Examples: real-time server push, occasional connectivity, and collaborative applications. Felx 2.0 has built-in support for publish/subscribe messaging and a new data services programming model, Flex Enterprise Services provides the following server-side data management channels:
* RPC Services provide traditional Flash Remoting RPC-style service invocation and managed proxying for HTTP and web service requests.
* Message Services enable applications using publish/subscribe messaging, collaboration, and real-time data push.
* Data Services enable you to more productively build applications that incorporate features such as data synchronization, transactional control and conflict management, automatic paging for large data sets, and support for application clustering.
http://labs.macromedia.com/
Flex Enterprise Services 2.0 enables a more productive way to build data-intensive Flex applications. Examples: real-time server push, occasional connectivity, and collaborative applications. Felx 2.0 has built-in support for publish/subscribe messaging and a new data services programming model, Flex Enterprise Services provides the following server-side data management channels:
* RPC Services provide traditional Flash Remoting RPC-style service invocation and managed proxying for HTTP and web service requests.
* Message Services enable applications using publish/subscribe messaging, collaboration, and real-time data push.
* Data Services enable you to more productively build applications that incorporate features such as data synchronization, transactional control and conflict management, automatic paging for large data sets, and support for application clustering.
Wednesday, January 04, 2006
The Nickull Threshold
Back to work for less than 48 hours after the holidays and I am already overwhelmed by the immense load of content being served up to me. The Web 2.0, RSS, Blogs, news, AJAX, Flash, HTML, click ads, pop-ups etc are all crying for my attention like a bunch of starving pigeons flocking to a park bench where the person is about to throw around bread crumbs. This got me thinking about a new term I am going to introduce and call the "Nickull Threshold".
The principle of the "Nickull Threshold" is to monitor the level of content being produced vs consumed. Note that when we all collectively produce more content than we can possibly consume during a specific frame of time, we have crossed the Nickull Threshold. At this point we have collectively diminished the usefulness of the internet and have collectively behaved irresponsibly given that on average no one will consume most of the content being created.
The formula:
The metric for the Nickull Threshold is an index NTi. Content Assimilated (CA) is divided by Content Published (CP) using an appropriate metric. For blog publishing, perhaps the rough measure is kilobytes. If I publish 1,000 kb in this blog and assimilate (read) 500 kb, my NTi is 500 / 1,000 or a value of .5. To achieve equilibrium, I must achieve a goal of 1.0 or higher for any specific frame. If I publish 1,000 kb of text and read 2,000 kb of text, I achieve a value of 2.
Look at this blog. Logically if I create one blog entry today, to do my part to remain under the Nickull Threshold, I must read one or more other blogs of equal size. If I only create this entry and read no others until I have made my next entry, I have crossed the line.
So what are the social implications of doing this? Everyone's talking but no one is listening. If we do not listen to each other for long enough, our own rhetoric spewing from our craniums into the netherworld becomes less and less relevant to the others given we no longer recognize what is important to them.
My Wishlist:
1. Radio DJ's should be forced to listen to double the amount of radio that they produce. Unfortunately, due to the quality of radio these days, this would not likely achieve anything other than a reduction in the number of DJ's which is probably a healthy thing IMO. Hey - you are not getting paid by the syllable!
2. News anchors MUST achieve a NTi of 5 or higher with their CA derived from facts only, free of opinions, adjectives, slants, spins, or anything other than truth. Yeah right - we'll see this sometime soon...
3. Anyone blogging must take a vow to read 2 or more other blogs for every blog entry they product. Automatic syndication of content to your blog is not allowed. At no point are the number of blog entries in your own blog allowed to exceed the number of comments they have received. This is a sanity check to ensure that your time blogging is well spent. (Note to self: check later to see I have more than 8 comments.)
4. Analysts probably have the highest NTi (good) given they have to read far more than they produce. Any analyst must maintain a NTi of 5 or more at all times. You are not allowed to read the same content twice nor does reading your own publications count. Each analyst MUST read at least 2 articles from other firms before writing on a subject themselves.
Anyone want anything else added?
The principle of the "Nickull Threshold" is to monitor the level of content being produced vs consumed. Note that when we all collectively produce more content than we can possibly consume during a specific frame of time, we have crossed the Nickull Threshold. At this point we have collectively diminished the usefulness of the internet and have collectively behaved irresponsibly given that on average no one will consume most of the content being created.
The formula:
The metric for the Nickull Threshold is an index NTi. Content Assimilated (CA) is divided by Content Published (CP) using an appropriate metric. For blog publishing, perhaps the rough measure is kilobytes. If I publish 1,000 kb in this blog and assimilate (read) 500 kb, my NTi is 500 / 1,000 or a value of .5. To achieve equilibrium, I must achieve a goal of 1.0 or higher for any specific frame. If I publish 1,000 kb of text and read 2,000 kb of text, I achieve a value of 2.
Look at this blog. Logically if I create one blog entry today, to do my part to remain under the Nickull Threshold, I must read one or more other blogs of equal size. If I only create this entry and read no others until I have made my next entry, I have crossed the line.
So what are the social implications of doing this? Everyone's talking but no one is listening. If we do not listen to each other for long enough, our own rhetoric spewing from our craniums into the netherworld becomes less and less relevant to the others given we no longer recognize what is important to them.
My Wishlist:
1. Radio DJ's should be forced to listen to double the amount of radio that they produce. Unfortunately, due to the quality of radio these days, this would not likely achieve anything other than a reduction in the number of DJ's which is probably a healthy thing IMO. Hey - you are not getting paid by the syllable!
2. News anchors MUST achieve a NTi of 5 or higher with their CA derived from facts only, free of opinions, adjectives, slants, spins, or anything other than truth. Yeah right - we'll see this sometime soon...
3. Anyone blogging must take a vow to read 2 or more other blogs for every blog entry they product. Automatic syndication of content to your blog is not allowed. At no point are the number of blog entries in your own blog allowed to exceed the number of comments they have received. This is a sanity check to ensure that your time blogging is well spent. (Note to self: check later to see I have more than 8 comments.)
4. Analysts probably have the highest NTi (good) given they have to read far more than they produce. Any analyst must maintain a NTi of 5 or more at all times. You are not allowed to read the same content twice nor does reading your own publications count. Each analyst MUST read at least 2 articles from other firms before writing on a subject themselves.
Anyone want anything else added?
Tuesday, December 20, 2005
Intelligent Design vs. Evolution (finally solved)!!
I saw a news article today talking about a judge who banned "intelligent design" in school curriculums:
http://today.reuters.com/news/newsarticle.aspx?type=topNews&storyid=2005-12-20T164925Z_01_EIC059334_RTRUKOC_0_US-LIFE-EVOLUTION.xml
It seems there are zealots on both sides of this argument who think their way is the only correct way. Well, I am going to end it once and for all on this blog. Here is the solution for the final end to this controversy in three easy steps:
1. Draft a MoU between both groups issuing an edict that "god created evolution". This dovetails nicely into both points of view without offending anyone.
2. Both sides need to note that this theory ties up all the loose ends and explains everything. No one need be upset anymore.
3. Let individuals believe what they want, keep religion out of schools and government and move on to more important problems to solve.
To offer more help, I have made this partial list of problems that these people can turn their attention to now that the religious debate over the origins of human life is officially over.
Problems that we need to solve:
1. Getting our fellow humans to be nicer to each other.
2. Ending the conflicts in various regions around the world.
3. Figuring out a good way to provide cheap, non-polluting, sustainable energy to the masses of the planet.
4. Showing compassion for our fellow man when he is in times of need.
5. (add your own here...)
Move along - nothing to see here anymore....
http://today.reuters.com/news/newsarticle.aspx?type=topNews&storyid=2005-12-20T164925Z_01_EIC059334_RTRUKOC_0_US-LIFE-EVOLUTION.xml
It seems there are zealots on both sides of this argument who think their way is the only correct way. Well, I am going to end it once and for all on this blog. Here is the solution for the final end to this controversy in three easy steps:
1. Draft a MoU between both groups issuing an edict that "god created evolution". This dovetails nicely into both points of view without offending anyone.
2. Both sides need to note that this theory ties up all the loose ends and explains everything. No one need be upset anymore.
3. Let individuals believe what they want, keep religion out of schools and government and move on to more important problems to solve.
To offer more help, I have made this partial list of problems that these people can turn their attention to now that the religious debate over the origins of human life is officially over.
Problems that we need to solve:
1. Getting our fellow humans to be nicer to each other.
2. Ending the conflicts in various regions around the world.
3. Figuring out a good way to provide cheap, non-polluting, sustainable energy to the masses of the planet.
4. Showing compassion for our fellow man when he is in times of need.
5. (add your own here...)
Move along - nothing to see here anymore....
Monday, December 12, 2005
Bio-mass: Sustainable Energy?
My Great Cousin Stig Nickull is the managing director at the world's largest (and most efficient) biomass energy conversion plant in Finland. The plant is in Pietarsaari and was specially designed to convert biomass fuel sources into electrical energy.
http://www.tekes.fi/opet/pdf/Alholma_2002.pdf
I write about this for two reasons. First and foremost, I am very proud of the heritage of my family and their contributions to the world. Stig is a brilliant scientist and seems to be causing positive change in the world. Imagine energy being freed from materials currently deemed garbage at an almost zero pollution factor? He has done it. We could definitely use this model in Canada since we have an abundance of wood chips, waste products etc.
The second is due to Carmen's comments about my hydrogen bashing on the previous blog entry:
http://technoracle.blogspot.com/2005/12/hydrogen-again-tweedle-dumb-and.html
Carmen points out that we need to change our thinking on this planet. She points out that these poor decisions are causing more problems and I agree. Politicians - try thinking - it doesn't hurt!!!!!
When evaluating alternative energy, please always question the Why, What, How and make sure you use the equation I wrote on the Hydrogen blog entry. We have to start thinking of solutions in terms of NET energy savings.
Peace to all!!
http://www.tekes.fi/opet/pdf/Alholma_2002.pdf
I write about this for two reasons. First and foremost, I am very proud of the heritage of my family and their contributions to the world. Stig is a brilliant scientist and seems to be causing positive change in the world. Imagine energy being freed from materials currently deemed garbage at an almost zero pollution factor? He has done it. We could definitely use this model in Canada since we have an abundance of wood chips, waste products etc.
The second is due to Carmen's comments about my hydrogen bashing on the previous blog entry:
http://technoracle.blogspot.com/2005/12/hydrogen-again-tweedle-dumb-and.html
Carmen points out that we need to change our thinking on this planet. She points out that these poor decisions are causing more problems and I agree. Politicians - try thinking - it doesn't hurt!!!!!
When evaluating alternative energy, please always question the Why, What, How and make sure you use the equation I wrote on the Hydrogen blog entry. We have to start thinking of solutions in terms of NET energy savings.
Peace to all!!
Tuesday, December 06, 2005
Hydrogen again - Tweedle Dumb and Tweedle Dumber!
I saw an article on truckers deciding to adopt hydrogen. As reported on Slashdot,
hipernoico writes to tell us Wired News is reporting that hundreds of semi trucks now on the roads are being partially powered by hydrogen. From the article: "These 18-wheelers make hydrogen as they go, eliminating the need for high-pressure, cryogenic storage tanks or hydrogen filling stations, which, by the way, don't yet exist. These truckers aren't just do-gooders. They like Canadian Hydrogen Energy's Hydrogen Fuel Injection, or HFI, system because it lets them save fuel, get more horsepower and, as a bonus, cause less pollution."
http://science.slashdot.org/article.pl?sid=05/11/15/2314241&from=rss
Let’s be straight about this. Using Hydrogen as a terrestrial fuel carrier is very stupid. The mathematics do not add up nor do the IQ points of those who continue to propose this for mainstream use. Any student who has passed Physics 101 knows this is true yet the dreamers seem to persist in disillusioning the masses about the benefits of using hydrogen.
The first rule of physics is that energy can be neither created nor destroyed. Energy may be converted from one form to another. A good fuel source for terrestrial use is one that occurs in abundance naturally, is easy to harvest, store and distribute and also converts cleanly with minimal impact on our environment. Coal, wood, oil all match some of this criteria. For example, wood is very abundant on our planet. It grows by converting the sun’s energy into biomass that may be easily harvested and transported then subsequently burned to release thermal energy. Thermal energy can be converted into kinetic energy very easily.
As with everything on this blog, do not take this for granted. Question it and do your own research.
So why is hydrogen as a fuel source for terrestrial application a bad idea? Let examine some facts and what hydrogen’s role is.
In most hydrogen powered vehicle scenarios, Hydrogen essentially acts as a battery or transporter of energy. It exists to store energy to allow it to be mobile. Similar to how recharge-able batteries work, it takes energy to produce hydrogen pure enough to be useful as a power source. At the target, the hydrogen’s stored energy is essentially converted to electrical energy, although it may also be converted to thermal energy which may be more efficient to convert to kinetic energy.
Hydrogen is highly unstable as a gas and does not readily occur in nature. Unlike hydrocarbons, which are relatively stable compounds, hydrogen will do everything it can dissipate or change state if released into the atmosphere of our little green planet. As such, pure hydrogen is extremely rare on this planet. Even if you found a reserve of hydrogen, you would still need to harvest and transport it to distribution centre’s which in itself may take more energy than the hydrogen you harvested had in it.
Let’s look at the pattern.
1. A natural process leaves energy stored in a manner that is easily harvested by human actors. Examples – the energy of the sun creates wind patterns on the surface of the planet which may be harvested by windmills or solar energy is converted via photoelectric arrays into electrical energy.
2. The energy is distributed in some form. Example - a power grid distributes electricity via wires or it may be used to charge a battery which may be transported physically to the site.
3. The energy is consumed. In reality – it is simply converted into some other form since energy can neither be created nor destroyed.
To make this work, the energy used to retrieve and initially convert the energy from its natural state (rE) plus the energy to refine, store, transport and distribute the energy to the point of consumption (tKw) must be less than the energy retrieved in step 3 when the energy is consumed (nKw). If rE + tKw > nKw, the process is a net waste of energy and this is before you consider the energy used to create the specialized mechanisms to consume an energy transport medium like hydrogen. Remember, fuel cells take more energy to design, manufacture, test and integrate into industry.
In general, we should not even consider anything less than a situation where
Hydrogen is not dense enough in a gaseous state to even contain enough energy to overcome the energy required to create, purify, store and transport it. Hydrogen as a terrestrial carrier of energy is a very dumb idea.
hipernoico writes to tell us Wired News is reporting that hundreds of semi trucks now on the roads are being partially powered by hydrogen. From the article: "These 18-wheelers make hydrogen as they go, eliminating the need for high-pressure, cryogenic storage tanks or hydrogen filling stations, which, by the way, don't yet exist. These truckers aren't just do-gooders. They like Canadian Hydrogen Energy's Hydrogen Fuel Injection, or HFI, system because it lets them save fuel, get more horsepower and, as a bonus, cause less pollution."
http://science.slashdot.org/article.pl?sid=05/11/15/2314241&from=rss
Let’s be straight about this. Using Hydrogen as a terrestrial fuel carrier is very stupid. The mathematics do not add up nor do the IQ points of those who continue to propose this for mainstream use. Any student who has passed Physics 101 knows this is true yet the dreamers seem to persist in disillusioning the masses about the benefits of using hydrogen.
The first rule of physics is that energy can be neither created nor destroyed. Energy may be converted from one form to another. A good fuel source for terrestrial use is one that occurs in abundance naturally, is easy to harvest, store and distribute and also converts cleanly with minimal impact on our environment. Coal, wood, oil all match some of this criteria. For example, wood is very abundant on our planet. It grows by converting the sun’s energy into biomass that may be easily harvested and transported then subsequently burned to release thermal energy. Thermal energy can be converted into kinetic energy very easily.
As with everything on this blog, do not take this for granted. Question it and do your own research.
So why is hydrogen as a fuel source for terrestrial application a bad idea? Let examine some facts and what hydrogen’s role is.
In most hydrogen powered vehicle scenarios, Hydrogen essentially acts as a battery or transporter of energy. It exists to store energy to allow it to be mobile. Similar to how recharge-able batteries work, it takes energy to produce hydrogen pure enough to be useful as a power source. At the target, the hydrogen’s stored energy is essentially converted to electrical energy, although it may also be converted to thermal energy which may be more efficient to convert to kinetic energy.
Hydrogen is highly unstable as a gas and does not readily occur in nature. Unlike hydrocarbons, which are relatively stable compounds, hydrogen will do everything it can dissipate or change state if released into the atmosphere of our little green planet. As such, pure hydrogen is extremely rare on this planet. Even if you found a reserve of hydrogen, you would still need to harvest and transport it to distribution centre’s which in itself may take more energy than the hydrogen you harvested had in it.
Let’s look at the pattern.
1. A natural process leaves energy stored in a manner that is easily harvested by human actors. Examples – the energy of the sun creates wind patterns on the surface of the planet which may be harvested by windmills or solar energy is converted via photoelectric arrays into electrical energy.
2. The energy is distributed in some form. Example - a power grid distributes electricity via wires or it may be used to charge a battery which may be transported physically to the site.
3. The energy is consumed. In reality – it is simply converted into some other form since energy can neither be created nor destroyed.
To make this work, the energy used to retrieve and initially convert the energy from its natural state (rE) plus the energy to refine, store, transport and distribute the energy to the point of consumption (tKw) must be less than the energy retrieved in step 3 when the energy is consumed (nKw). If rE + tKw > nKw, the process is a net waste of energy and this is before you consider the energy used to create the specialized mechanisms to consume an energy transport medium like hydrogen. Remember, fuel cells take more energy to design, manufacture, test and integrate into industry.
In general, we should not even consider anything less than a situation where
Hydrogen is not dense enough in a gaseous state to even contain enough energy to overcome the energy required to create, purify, store and transport it. Hydrogen as a terrestrial carrier of energy is a very dumb idea.
Monday, November 28, 2005
Introducing Serena!
My second daughter was born Friday November 18 at 18:39 Pacific time. At birth she weighed in at 4.3 Kg (about 9 lbs, 5 ounces for those of you who keep the metric system supressed). Here she is at home a week later being held by her big sister Bianca. I am the luckiest man in the world - what can I say!
A photo of two angels...
A photo of two angels...
Friday, November 11, 2005
What is meant by “Service Opacity” and “Managed Transparency”?
Okay – so you all know about Service Oriented Architecture (SOA) and have your ideas of what it is. While we all are likely to disagree with the finer points of SOA, most will agree on a few core tenets of SOA. If you want to discuss this with me, the standard contribution of a vintage Bordeaux applies.
Core to SOA is the existence of services as autonomous entities that act as a boundary between some “functionality” and the entity that consumes that functionality via the service. Service implementers should take care to design services as opaque as possible. This means that as a service consumer, you should talk to the service, but not really care about how the service implements the functionality it provides. This Black Box aspect of SOA is really a specialized notion of the definition of software architecture in the great green book “Documenting Software Architectures: Views and Beyond” by Clements, Bachman et al. In this book, software architecture is defined as “the structure or structures of the system, which consist of elements and their externally visible properties, and the relationships among them”. The key words here are “externally visible properties”. A service provider adhering to this basic axiom should strive to only reveal externally visible properties that are critical for the consumer to know, but no more.
Critique: That is a broad statement. Why? As with anything in technology, don’t run out and do it just because I said so. Question everything. Critique this if you don’t agree.
Rationale: Services provide a healthy abstraction between functionality and those using the functionality. By not revealing the specific implementation behind the service, no consumer of that service has to create a tighter integration than necessary. This frees the service provider to implement, maintain, replace or update the functionality behind the service with the least amount of concern for dependencies from consumers. As long as the replacement, update still supports the existing service interface, the consumer will not notice changes. Please note that I qualified my statement with “than necessary” – the exact level of this is specific to each implementation and it is unlikely anyone could create a hard and fast rule for this.
Specific details of how the service is fulfilling its’ functions are generally not relevant to the consumer either. One should not make assumptions about specific interactions that may happen behind the service and only stick to the externally visible real world effect (RWE). A service is a black box with an interface. As long as the interface allows you to do what you want, you shall be happy. Like this blog, all you need to know is that you enter a URI into the location window of your browser and you get this content back, formatted in HTML. You should not care whether or not this is static text residing on an Apache server or if I just typed into a blackberry really fast to give you this real time.
Example:
Some people have advocated that a service must provide a guarantee of delivery assurances between the service itself and the ultimate (application) destination. This delivery assurance effectively suggest that a service must deliver messages it receives to another destination, possibly with additional guarantees for “in order” and “eliminate duplicates” amongst other functions. I strongly disagree with this for a number of reasons. Is this really an externally visible property of a service we should expose? Perhaps in some cases it can be justified but I have trouble making any normative statements for such that apply to all cases. A functionality similar to this may be a higher level matter which I will try to explain at the end of my rant below.
Rant:
First, even specifying in a standard or protocol that there is an “application destination” behind a service is errant since it implies a specific model for infrastructure to be implemented behind the service interface. If this were accepted within the standard, does it imply that all implementations must now have a specific architecture where nothing can be processed locally on the service?
Secondly, assuming you are going to send a sequence of six messages to a specific service and invoke a delivery assurance/guarantee that all messages in that sequence are delivered “in order”, this now implies a cardinality of 1:1 between a service and the “thing” that must process the messages. Does this disallow the service itself to process part of the message?
Third, this model constrains implementers from dividing up an incoming message to distribute it to several processing classes or applications. The mechanism as it stands today essentially requires a sequence number for each incoming message which must monotonically increase by a factor of one. The intentions are that the service will forward incoming messages to the application destination in the same order it received them in. A cardinality of 1:1 is sub-optimal for scaling IMO. What about pooling? Does this count as one application or several? And why should the service consumer care about this?
The UML model above captures the implied infrastructure. It implies that all incoming messages in a sequence flagged with “in order” delivery MUST be forwarded intact as whole units to one application destination. Why should an implementer not be able to forward messages 1 and 3 to one application destination and messages 4 and 6 to another then simply coordinate those activities to ensure they are “processed” in order? Is the following second diagram below illegal if a guarantee of “in order” delivery is requested? Note that in the following diagram, I have replaced Service.forward() with Service.coordinateProcessing() since this is really what the service consumer is after. After all, even if messages *are* delivered “in order”, there is no guarantee that they have been processed in order.
Note: Please excuse my poor UML style of multiple ApplicationDestination classes but I felt compelled to depict it this way to illustrate the splitting of various messages to different endpoints.
An operation such as “forward()” (from the first diagram) should not be a mandatory externally visible property of a service. It implies a specific model for implementing a service and any specifications that deal with service interoperability should not try to impose specific implementation styles on service providers. My argument does not mean that implementers cannot do this if they see fit or determine that it is necessary, but this is not a one size shoe that should force all to fit within.
A higher level look at the goal.
The real goal is to allow a service consumer to flag that a sequence of messages are meant to be “processed sequentially”. It should not care how that is done, just that this is done or, in the alternative, the service generates a fault. What mechanisms do we need to do this?
1. We need to ensure that the message on the wire between the service consumer and the service carries with it all the tokens to enable the service to understand what type of serial/cardinal processing assurances it requires.
2. We also need a messaging mechanism that allows a service to reassemble the complete stream representing the message in a deterministic manner matching the stream exactly as it was when it left the service consumer.
3. Another requirement is that services must have some form of service description that allows them to declare what their exact capabilities are WRT serial processing of multiple calls.
Does it look something like this?
Note that the cardinality between Service and ApplicationDestination is “any”. It is possible that a service may simply process an invocation request locally. Not all attributes and operations have been explicitly called out to avoid confusion but hopefully this will give you the idea.
Conclusion:
Managed transparency should be a core consideration of a service provider. Services should remain as opaque as possible but may need to expose some aspects of their operations to facilitate their use but consumers. Those implementing services must take both of these into account lest we risk building a world wide network of interdependent, tightly bound applications – the very thing that SOA is attempting to save us of.
What do you think?
Core to SOA is the existence of services as autonomous entities that act as a boundary between some “functionality” and the entity that consumes that functionality via the service. Service implementers should take care to design services as opaque as possible. This means that as a service consumer, you should talk to the service, but not really care about how the service implements the functionality it provides. This Black Box aspect of SOA is really a specialized notion of the definition of software architecture in the great green book “Documenting Software Architectures: Views and Beyond” by Clements, Bachman et al. In this book, software architecture is defined as “the structure or structures of the system, which consist of elements and their externally visible properties, and the relationships among them”. The key words here are “externally visible properties”. A service provider adhering to this basic axiom should strive to only reveal externally visible properties that are critical for the consumer to know, but no more.
Critique: That is a broad statement. Why? As with anything in technology, don’t run out and do it just because I said so. Question everything. Critique this if you don’t agree.
Rationale: Services provide a healthy abstraction between functionality and those using the functionality. By not revealing the specific implementation behind the service, no consumer of that service has to create a tighter integration than necessary. This frees the service provider to implement, maintain, replace or update the functionality behind the service with the least amount of concern for dependencies from consumers. As long as the replacement, update still supports the existing service interface, the consumer will not notice changes. Please note that I qualified my statement with “than necessary” – the exact level of this is specific to each implementation and it is unlikely anyone could create a hard and fast rule for this.
Specific details of how the service is fulfilling its’ functions are generally not relevant to the consumer either. One should not make assumptions about specific interactions that may happen behind the service and only stick to the externally visible real world effect (RWE). A service is a black box with an interface. As long as the interface allows you to do what you want, you shall be happy. Like this blog, all you need to know is that you enter a URI into the location window of your browser and you get this content back, formatted in HTML. You should not care whether or not this is static text residing on an Apache server or if I just typed into a blackberry really fast to give you this real time.
Example:
Some people have advocated that a service must provide a guarantee of delivery assurances between the service itself and the ultimate (application) destination. This delivery assurance effectively suggest that a service must deliver messages it receives to another destination, possibly with additional guarantees for “in order” and “eliminate duplicates” amongst other functions. I strongly disagree with this for a number of reasons. Is this really an externally visible property of a service we should expose? Perhaps in some cases it can be justified but I have trouble making any normative statements for such that apply to all cases. A functionality similar to this may be a higher level matter which I will try to explain at the end of my rant below.
Rant:
First, even specifying in a standard or protocol that there is an “application destination” behind a service is errant since it implies a specific model for infrastructure to be implemented behind the service interface. If this were accepted within the standard, does it imply that all implementations must now have a specific architecture where nothing can be processed locally on the service?
Secondly, assuming you are going to send a sequence of six messages to a specific service and invoke a delivery assurance/guarantee that all messages in that sequence are delivered “in order”, this now implies a cardinality of 1:1 between a service and the “thing” that must process the messages. Does this disallow the service itself to process part of the message?
Third, this model constrains implementers from dividing up an incoming message to distribute it to several processing classes or applications. The mechanism as it stands today essentially requires a sequence number for each incoming message which must monotonically increase by a factor of one. The intentions are that the service will forward incoming messages to the application destination in the same order it received them in. A cardinality of 1:1 is sub-optimal for scaling IMO. What about pooling? Does this count as one application or several? And why should the service consumer care about this?
The UML model above captures the implied infrastructure. It implies that all incoming messages in a sequence flagged with “in order” delivery MUST be forwarded intact as whole units to one application destination. Why should an implementer not be able to forward messages 1 and 3 to one application destination and messages 4 and 6 to another then simply coordinate those activities to ensure they are “processed” in order? Is the following second diagram below illegal if a guarantee of “in order” delivery is requested? Note that in the following diagram, I have replaced Service.forward() with Service.coordinateProcessing() since this is really what the service consumer is after. After all, even if messages *are* delivered “in order”, there is no guarantee that they have been processed in order.
Note: Please excuse my poor UML style of multiple ApplicationDestination classes but I felt compelled to depict it this way to illustrate the splitting of various messages to different endpoints.
An operation such as “forward()” (from the first diagram) should not be a mandatory externally visible property of a service. It implies a specific model for implementing a service and any specifications that deal with service interoperability should not try to impose specific implementation styles on service providers. My argument does not mean that implementers cannot do this if they see fit or determine that it is necessary, but this is not a one size shoe that should force all to fit within.
A higher level look at the goal.
The real goal is to allow a service consumer to flag that a sequence of messages are meant to be “processed sequentially”. It should not care how that is done, just that this is done or, in the alternative, the service generates a fault. What mechanisms do we need to do this?
1. We need to ensure that the message on the wire between the service consumer and the service carries with it all the tokens to enable the service to understand what type of serial/cardinal processing assurances it requires.
2. We also need a messaging mechanism that allows a service to reassemble the complete stream representing the message in a deterministic manner matching the stream exactly as it was when it left the service consumer.
3. Another requirement is that services must have some form of service description that allows them to declare what their exact capabilities are WRT serial processing of multiple calls.
Does it look something like this?
Note that the cardinality between Service and ApplicationDestination is “any”. It is possible that a service may simply process an invocation request locally. Not all attributes and operations have been explicitly called out to avoid confusion but hopefully this will give you the idea.
Conclusion:
Managed transparency should be a core consideration of a service provider. Services should remain as opaque as possible but may need to expose some aspects of their operations to facilitate their use but consumers. Those implementing services must take both of these into account lest we risk building a world wide network of interdependent, tightly bound applications – the very thing that SOA is attempting to save us of.
What do you think?
Wednesday, November 09, 2005
Any minute now...
Yes - my wife is *very* pregnant. We are due to give birth any second. Since I have been innundated with requests for photos, I decided to make a short blog entry to illustrate exactly why I am the luckiest man alive. Our official due date is November 11th (two days from now).
It is hard to believe Bianca is now 2 and a half years old. Time flies.
Monday, November 07, 2005
Anti-patterns for SOA: Part Two
Three general rules for service granularity and redundancy
I recently made a blog entry about “anti-patterns” for SOA. This is funny since I never took the time to define either "SOA" or “anti-patterns” at the beginning of the blog which lead to almost everyone reading it walking away with a different opinion. I say funny because I am the sort of person who finds that type of thing amusing ;-)
Okay – seriously, before I continue, I would like to put some definitions around both of these. For SOA, I will rely on the OASIS Service Oriented Architecture Reference Model Technical Committee’s work to develop a Reference Model for SOA. While only an editors’ draft at this time, the basic premise is that SOA is an architectural paradigm or simply stated – a way of viewing/doing things that center’s around the concept of a service. Revision 10 is coming out soon.
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=soa-rm
SOA definitions are like a tush. Everyone has one. Anti-pattern definitions are similar. To lexically scope the rest of this blog, I will rely on the following definition introduced by Hayes McCormick, a lead engineer from Mitre:
“Anti patterns are negative patterns that present more problems that they produce”
http://www.antipatterns.com/briefing/sld003.htm
Hays also compares various types of anti-patterns. In IT, anti pattern examples may be the god class ( one large class that does everything), spaghetti code (procedural and un-ordered/unstructured), Design by Committee (always problematic) and those of you who use vi instead of emacs or pico (miscreants – you know who you are!!). Social anti-patterns include terrorism, pyromania, drug abuse and those or you who use vi instead or emacs of pico.
Rule 1: Granularity is in the eye of the beholder.
The question of service granularity seems to be a pervasive and confusing topic. Someone recently told me that an SOA with ten million services is probably a bad practice since it has too large a number of services and hence is hard to manage. I argued that number of services as a sole metric for whether or not SOA is implemented correctly is not a good metric given that it is subjective to the eye of the beholder. The point is that the rationalization for these things is not always obvious to the viewer.
Case in point? How about the Internet. The internet has over ten million individual services yet each of those services have a unique and valid function, at least to the owners of the content and those wishing to retrieve it. So look at this in terms of granularity. Should clusters of those services be bundled together? No. You are reading this blog which aggregates a few things (my posts plus thousands of comments calling for my banishment for heresy), however I will argue that IMO this blog should remain its’ own service.
If someone also owned a distributed grid computing network, they may wish to allow consumers to make leases to individual nodes via services. This means that each node in the cluster would have at least one service. The alternative is to implement an uber-service that then locates and leases the appropriate node on the cluster on behalf of the consumer and forces the consumer to remain agnostic with respect to the physical contract to the cluster node. So which is valid? Well, both. There are advantages to doing it each way.
So how far could you take the granularity/number of services concept before it becomes a legitimate anti-pattern? In J2EE many have made assertions about patterns and anti-patterns. The issue of granularity is measured by java programmers in terms of what they deem efficient or justifiable based on expected overhead. Puneet M. Sangal wrote:
“Accessing Fine-grained EJB Interfaces
Marshaling and unmarshalling data involved in a remote call across the network can cause major time latency in applications. Latency can also occur due to multiple remote calls. Based on the needs on the application, the solution might involve redesigning the EJBs, but if that's not the case, servlets should not remotely access an EJB entity via a fine-grained API. If the EJB's expose both fine and coarse-grained APIs then servlets should use single calls to the coarse-grained API in preference to making multiple calls to the fine-grained methods.”
Is this view justified? IMO, it gives us hints about some concepts upon which to base our decisions.
Convoluted SOA Contest Winning Idea?
Let’s take this to an extreme case. Imagine you wanted to render a raster graphic of some new platform. Let’s assume that the native format for the graphic returned a large hash of pixel values in Hex format. I will use text to make this easy to understand:
1, 1, #C0C0FF //pixel located at quadrant 1 from top, 1 from left, color is Hex #C0C0FF
1, 2, #C0C0C0
1, 3, #FFFFFF
Etc.
Now imagine that each pixel’s value had to be parameterized to web service call that would return the Hex value in RGB values. We can make this more granular by forcing the consumer to call a different service for each Hex value. Accordingly, the service endpoints would be http://www.domain-example.com:port/[hex_value]/ws
If you sent in #FFFFFF, the response would be rgb(255,255,255) which would render white. It is possible to make rendering software for your new raster graphic format make one service call for each pixel in the image.
I probably don’t have to tell you that this is a bad idea. It is very black and white (no irony intended) however there are other examples that are less ambiguous.
Rule 2: It is probably not possible to write a rule for granularity that is applicable to all situations.
For the record, the preceding idea was not original, it is merely a new twist on an old ploy to make a picture using an HTML table with hundreds of rows and columns, each having its’ own background color. Collectively, the table cells being rendered in an HTML client application would look like a picture. Hey – maybe we could produce that table by linking up the results of all the service calls……? Somebody shoot me now!!
In the preceding case, it would have made sense to call a single service that returned a large lookup table for all values, then simply iterate each Hex value against that for the corresponding value locally. My gut instinct would have been to architect it this way, but others may object (for various and perhaps legitimate reasons).
There are several questions that I will pose to you on this subject, interspersed with my answers. YAMMV (Your actual mileage may vary):
1. Should service calls be used in places where the overhead of incurring them exceeds the resources required to enable the functionality locally?
[Duane]: My gut reaction is that this is a consideration but will not always be the largest factor in the equation.
2. Would it be prudent to state that the call to the service should be coded in less lines that it would take to reproduce the functionality locally. For example, making a call to multiply two numbers together may take 25 lines of code in Java, including wrapping it in a “try/catch” construct. Doesn’t it always make more sense to simply write:
…
int var1;
int var2;
int result;
try {
result = var1 * var2;
}
catch (Exception e) {
//do something with error here…
}
…
[Duane]: Many programmers will probably not make remote calls for simple functions they can easily write. I am not sure if any rule can be made from this.
3. Is there any chance that “catch all” rules can be made to determine whether or not services are too granular? Consider the fact that some programming languages allow for sentences to be of type “String” and also allow them to be treated as an array of chars.
[Duane]: I do not believe this will be possible given we all tend to look at problems form different aspects.
4. If any of these rules are violated, is the thing still SOA?
[Duane]: SOA is SOA, whether good or bad design.
All of this is dependent upon your own views and principles.
I started thinking about this in terms of how other related efforts handle this. The first thing that struck me is the use of inheritance in object oriented programming has a strong effect on whether or not you use a local method or make a remote call. For example, imagine the following example as a class hierarchy.
If you need to do a multiplication operation on two or more integers, you would logically use the Calculus class. If you are writing a new class, you would import the Calculus class then call the multiply() method and feed it parameters. This would return a result. Importing the class gives you access to all the methods that class has.
It would be highly illogical to make a remote procedure call to another class for some functionality your imported class already possesses such as the multiply() method.
You will notice that in the hierarchy I established above, there are two methods on the language side that are very similar – WriteBookReport() in the English Literature class and writeReport() in the Grammar class. This illustrates another problem that exists within SOA – the number of similar or overlapping services. This has also been raised as a possible anti-pattern of SOA by a number of people.
In this case, the designer of the overall programming language would have the ability to re-architect the class hierarchy to form a more generic operation that may be called “writeReport()” and take a parameter of reportType (which might have a value of “book report” in certain instances).
This often has not been addressed within the context of SOA. class hierarchies are commonplace in every modern language. Service classification hierarchies are not as common. This may be an interesting thing to look at.
Conceptually, it the writeReport() operation the same as the writeBookReport() operation? This depends upon your perspective on the problem and the granularity of which you examine the operation.
A simple table with some generic aspects highlighted seems to indicate that the two operations are the same:
The two operations seem pretty similar from this perspective. Of course, when you examine other aspects, this starts to fall apart.
Okay – so this example was a little daft. Let’s look at a more relevant issue. Is an HTTP get() different from an HTTP post()? The patterns are both the same. You marshall a message into text and route it over the internet using an established protocol. It reaches a service which then evaluates the message and forms a response message which is then routed back to the sender. They both do this. So do the HTTP put(), delete() and other operations. Where they differ in functionality is not at the wire level, it is at the conceptual or real world effect level. This is an important consideration for SOA. What is the real world effect of invoking the service.
Rule 3: Duplication of Services is in the eye of the beholder.
Conclusion:
Alongside the issue of service granularity, we have a similar issue of duplication of services. Both of these issues are probably beyond some form of an immediate “best practices” guide that is applicable in all situations.
What do you think?
I recently made a blog entry about “anti-patterns” for SOA. This is funny since I never took the time to define either "SOA" or “anti-patterns” at the beginning of the blog which lead to almost everyone reading it walking away with a different opinion. I say funny because I am the sort of person who finds that type of thing amusing ;-)
Okay – seriously, before I continue, I would like to put some definitions around both of these. For SOA, I will rely on the OASIS Service Oriented Architecture Reference Model Technical Committee’s work to develop a Reference Model for SOA. While only an editors’ draft at this time, the basic premise is that SOA is an architectural paradigm or simply stated – a way of viewing/doing things that center’s around the concept of a service. Revision 10 is coming out soon.
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=soa-rm
SOA definitions are like a tush. Everyone has one. Anti-pattern definitions are similar. To lexically scope the rest of this blog, I will rely on the following definition introduced by Hayes McCormick, a lead engineer from Mitre:
“Anti patterns are negative patterns that present more problems that they produce”
http://www.antipatterns.com/briefing/sld003.htm
Hays also compares various types of anti-patterns. In IT, anti pattern examples may be the god class ( one large class that does everything), spaghetti code (procedural and un-ordered/unstructured), Design by Committee (always problematic) and those of you who use vi instead of emacs or pico (miscreants – you know who you are!!). Social anti-patterns include terrorism, pyromania, drug abuse and those or you who use vi instead or emacs of pico.
Rule 1: Granularity is in the eye of the beholder.
The question of service granularity seems to be a pervasive and confusing topic. Someone recently told me that an SOA with ten million services is probably a bad practice since it has too large a number of services and hence is hard to manage. I argued that number of services as a sole metric for whether or not SOA is implemented correctly is not a good metric given that it is subjective to the eye of the beholder. The point is that the rationalization for these things is not always obvious to the viewer.
Case in point? How about the Internet. The internet has over ten million individual services yet each of those services have a unique and valid function, at least to the owners of the content and those wishing to retrieve it. So look at this in terms of granularity. Should clusters of those services be bundled together? No. You are reading this blog which aggregates a few things (my posts plus thousands of comments calling for my banishment for heresy), however I will argue that IMO this blog should remain its’ own service.
If someone also owned a distributed grid computing network, they may wish to allow consumers to make leases to individual nodes via services. This means that each node in the cluster would have at least one service. The alternative is to implement an uber-service that then locates and leases the appropriate node on the cluster on behalf of the consumer and forces the consumer to remain agnostic with respect to the physical contract to the cluster node. So which is valid? Well, both. There are advantages to doing it each way.
So how far could you take the granularity/number of services concept before it becomes a legitimate anti-pattern? In J2EE many have made assertions about patterns and anti-patterns. The issue of granularity is measured by java programmers in terms of what they deem efficient or justifiable based on expected overhead. Puneet M. Sangal wrote:
“Accessing Fine-grained EJB Interfaces
Marshaling and unmarshalling data involved in a remote call across the network can cause major time latency in applications. Latency can also occur due to multiple remote calls. Based on the needs on the application, the solution might involve redesigning the EJBs, but if that's not the case, servlets should not remotely access an EJB entity via a fine-grained API. If the EJB's expose both fine and coarse-grained APIs then servlets should use single calls to the coarse-grained API in preference to making multiple calls to the fine-grained methods.”
Is this view justified? IMO, it gives us hints about some concepts upon which to base our decisions.
Convoluted SOA Contest Winning Idea?
Let’s take this to an extreme case. Imagine you wanted to render a raster graphic of some new platform. Let’s assume that the native format for the graphic returned a large hash of pixel values in Hex format. I will use text to make this easy to understand:
1, 1, #C0C0FF //pixel located at quadrant 1 from top, 1 from left, color is Hex #C0C0FF
1, 2, #C0C0C0
1, 3, #FFFFFF
Etc.
Now imagine that each pixel’s value had to be parameterized to web service call that would return the Hex value in RGB values. We can make this more granular by forcing the consumer to call a different service for each Hex value. Accordingly, the service endpoints would be http://www.domain-example.com:port/[hex_value]/ws
If you sent in #FFFFFF, the response would be rgb(255,255,255) which would render white. It is possible to make rendering software for your new raster graphic format make one service call for each pixel in the image.
I probably don’t have to tell you that this is a bad idea. It is very black and white (no irony intended) however there are other examples that are less ambiguous.
Rule 2: It is probably not possible to write a rule for granularity that is applicable to all situations.
For the record, the preceding idea was not original, it is merely a new twist on an old ploy to make a picture using an HTML table with hundreds of rows and columns, each having its’ own background color. Collectively, the table cells being rendered in an HTML client application would look like a picture. Hey – maybe we could produce that table by linking up the results of all the service calls……? Somebody shoot me now!!
In the preceding case, it would have made sense to call a single service that returned a large lookup table for all values, then simply iterate each Hex value against that for the corresponding value locally. My gut instinct would have been to architect it this way, but others may object (for various and perhaps legitimate reasons).
There are several questions that I will pose to you on this subject, interspersed with my answers. YAMMV (Your actual mileage may vary):
1. Should service calls be used in places where the overhead of incurring them exceeds the resources required to enable the functionality locally?
[Duane]: My gut reaction is that this is a consideration but will not always be the largest factor in the equation.
2. Would it be prudent to state that the call to the service should be coded in less lines that it would take to reproduce the functionality locally. For example, making a call to multiply two numbers together may take 25 lines of code in Java, including wrapping it in a “try/catch” construct. Doesn’t it always make more sense to simply write:
…
int var1;
int var2;
int result;
try {
result = var1 * var2;
}
catch (Exception e) {
//do something with error here…
}
…
[Duane]: Many programmers will probably not make remote calls for simple functions they can easily write. I am not sure if any rule can be made from this.
3. Is there any chance that “catch all” rules can be made to determine whether or not services are too granular? Consider the fact that some programming languages allow for sentences to be of type “String” and also allow them to be treated as an array of chars.
[Duane]: I do not believe this will be possible given we all tend to look at problems form different aspects.
4. If any of these rules are violated, is the thing still SOA?
[Duane]: SOA is SOA, whether good or bad design.
All of this is dependent upon your own views and principles.
I started thinking about this in terms of how other related efforts handle this. The first thing that struck me is the use of inheritance in object oriented programming has a strong effect on whether or not you use a local method or make a remote call. For example, imagine the following example as a class hierarchy.
If you need to do a multiplication operation on two or more integers, you would logically use the Calculus class. If you are writing a new class, you would import the Calculus class then call the multiply() method and feed it parameters. This would return a result. Importing the class gives you access to all the methods that class has.
It would be highly illogical to make a remote procedure call to another class for some functionality your imported class already possesses such as the multiply() method.
You will notice that in the hierarchy I established above, there are two methods on the language side that are very similar – WriteBookReport() in the English Literature class and writeReport() in the Grammar class. This illustrates another problem that exists within SOA – the number of similar or overlapping services. This has also been raised as a possible anti-pattern of SOA by a number of people.
In this case, the designer of the overall programming language would have the ability to re-architect the class hierarchy to form a more generic operation that may be called “writeReport()” and take a parameter of reportType (which might have a value of “book report” in certain instances).
This often has not been addressed within the context of SOA. class hierarchies are commonplace in every modern language. Service classification hierarchies are not as common. This may be an interesting thing to look at.
Conceptually, it the writeReport() operation the same as the writeBookReport() operation? This depends upon your perspective on the problem and the granularity of which you examine the operation.
A simple table with some generic aspects highlighted seems to indicate that the two operations are the same:
The two operations seem pretty similar from this perspective. Of course, when you examine other aspects, this starts to fall apart.
Okay – so this example was a little daft. Let’s look at a more relevant issue. Is an HTTP get() different from an HTTP post()? The patterns are both the same. You marshall a message into text and route it over the internet using an established protocol. It reaches a service which then evaluates the message and forms a response message which is then routed back to the sender. They both do this. So do the HTTP put(), delete() and other operations. Where they differ in functionality is not at the wire level, it is at the conceptual or real world effect level. This is an important consideration for SOA. What is the real world effect of invoking the service.
Rule 3: Duplication of Services is in the eye of the beholder.
Conclusion:
Alongside the issue of service granularity, we have a similar issue of duplication of services. Both of these issues are probably beyond some form of an immediate “best practices” guide that is applicable in all situations.
What do you think?
Friday, November 04, 2005
Anti-patterns for SOA: Part One
This is a quick paper to explain some of the core tenets of SOA and illustrate a potentially unhealthy practice of trying to make functionality that exists behind a service interface visible to service consumers. Sadly, this practice seems to be growing in popularity lately which makes me question if people really understand why we are trying to move to SOA in the first place.
I came across this situation as editor of a requirements document for what may be a service oriented architecture for content management. In an early draft, I had copied what someone wrote:
"..a derived requirement that services would need to exist for Search, Read, Write, Copy, Move, Update, Merge, Version, Lock, Access Control Introspect type methods for content management. "
I received a comment back that stated :
“Content transformation – might be overloaded on copy… alternatively might want to be able to create complex transformation pipelines (splits, branches and merge…)“
It turns out that this insightful comment was extrenmely useful and lead to an epiphany. I now believe that the set should be shortened to only the following services:
Search – to locate a reference from which the content may be retrieved.
Read/Retrieve – essentially the ability to retrieve a copy of content.
Write – a method that would make subsequent retrieval of the content via a service possible. Note that I did not state „to write it to some persistent media“. This will be explained more later.
Update – a method to update content without over-writing an existing version. This assumes some form of version control.
Move – the ability to associate content with an additional URI, IRI or similar reference pointer. Note that I did not state „to migrate content from one persistent location to another“. That makes assumptions of some specific functionality or mechanism behind a service interface.
Lock – lock permissions for content so no one else can invoke the other methods.
Delete – make the content no longer reference able via the service interface. This does not necessarily imply removing the binary sequence that represents the content.
Access Control Introspect – inspecting the access control policies associated with content.
The following should not be core services, but should be optional:
Copy – essentially, this is akin to a combination of read and a subsequent write. This could optionally be implemented but should not be core since it is redundant.
Move – implies the ability to migrate content from one persistent location to another and is nothing more that a combination of retrieve, write and delete. Note that in some cases, content may not actually physically move, it may simply just have a different reference. Many operating systems work this way.
Merge – merge is really retrieval of two or more bits of content, a port-retrieval join and then an write() of a new aggregate content. It can also take place entirely behind the service interface by the creation of a method that will enable the end result of the merge to be retrieved from a service call. This does not mean that the content is actually merged – it is simply possible to get aggregated content.
So you are probably seeing a pattern by now. The core tenet is the externally visible properties or effects of invoking an action VS. the functionality that enables the end result. In SOA, one should not try to focus on the latter, the focus is brought to the service itself. The implementation is not relevant to the consumer.
Case in point. Microsoft engineers have engineered windows so via the user interface, it appears that you can move something from one location on a hard disk to another. But did it really move? No. In fact, in many cases, all you are really doing is changing the pointers which results in a change in the way the view is created for the user via the GUI. Those of you who have programmed in C and CPP will be familiar with this concept.
One service should probably be out of scope for the first phase of the project:
Transform – implying custom manipulations of content.
So why should the first group be core and transform be not in scope?
Rationale:
Services are autonomous and manage their transparency. A service should remain as opaque as possible to mitigate creating dependencies from those who invoke the service. When you make a service request, you enter parameters based on what you want. If for example, a blob of data exists in XML format (let’s call it FOO), and you desire a copy of it in HTML format, you should ask for it like this:
Please retrieve FOO for me in HTML format?
Or
get(FOO, html);
may represent a call to a java class. FOO in this case represents the thing you want and html may be an optional parameter to force the class to provide it in html rather than some default format.
When invoking this method, you should not care how the method gets you FOO in html format. It is not your job to question that. You only care about one thing. It either gives you FOO in html or it fails. Whether or not FOO.html was created by applying a complex transformation, aggregation or forced composition is irrelevant in most cases. Perhaps monkey’s banging out machine byte code randomly created it for you. It really doesn’t matter.
Bad Practice:
You should not be forced to ask for it in any manner that attempts to control processes behind the service interface like this:
“Please retrieve FOO for me and transform it from XML to HTML.”
The latter is bad architecture since it introduces unnecessary dependencies. When you ask for FOO in HTML format, you should not care whether or not it persists in HTML or some other format, only that you get what the interface promises you can retrieve.
The same principle applies for versioning and many other functions. Versions are merely parameters that can be entered into the server, but you should not specify how the functionality behind the service interface evaluates which one is the latest version or how it finds and serializes a specific version. All you need to know is that you either get the version you requested or a notice that the service failed.
Resolution:
There are multiple correct ways to architect a solution for allowing transformations. The first is based on the transformation happening behind the service interface. The service description can declare this is happening but should only do this if that information is necessary for the service consumer to know. Keep in mind that the Service Consumer cannot and should not see anything behind the service interface.
Example One – Good Practice:
Explanation:
The service consumer makes a call to the service and asks for a specific chunk of content in a specific format. In this case, it asks for FOO and specifies a parameter stating what format it wants foo in (HTML). The service takes the request, then retrieves a copy of FOO in xml format. Before passing it back to the service consumer, it calls an internal (and invisible to the service consumer) method to transform foo.xml into foo.html. It gets back the result then passes it back to the service consumer.
In this diagram, everything above the service consumer or below the service itself should be out of scope since there are no externally visible properties nor should assumptions be made about them. To introduce these dependencies would introduce unnecessary dependencies and assumptions and violate the core axioms of SOA.
Example Two – Good Practice
The second variation involves post–retrieval transformations and is also acceptable. Note that the patterns of interaction between service consumer and service provider are not broken.
In this case, the service is only capable of retrieving the FOO content in xml and passes the xml back to the service consumer. The service consumer subsequently transforms it to the HTML format.
In this scenario, the post retrieval use of the content is clearly out of scope for this particular SOA since it’s job is to facilitate the retrieval. There are an unlimited number of things that the user of the service consumer may do once it has retrieved the content. In Phase II, we MAY tackle a few of these however at present, all post retrieval uses should be out of scope until we have the core layer defined since there is a dependency on the base layer.
It should be noted that in this example, it is possible the Transformation component’s interface is visible to the service. That leads us to the third example.
Example Three – suboptimal
In this final scenario, the service consumer calls the service, and requests that the service redirect the response to a transformation service, with instructions to tell the transformation service to do something with it then return the result back to the service consumer.
This introduces several things that I would fire an architect for.
First – it mixes two tasks into one process. If something goes wrong, the service consumer will have great difficulty knowing where things went wrong. The service consumer also cannot accurately know the state of the service call since it would likely have no idea when the service passed the work to the transformation component.
Secondly, it also blurs two unique functions into one. This lessens the probability of uncomplicated reuse but building unnecessarily complex interfaces. The service interface, instead of handling simple requests, now has to handle requests overloaded with details of where to forward things, what protocol to use and what instructions to forward. Inelegant.
This scenario also introduces another potentially unhealthy risk – the service does not know if the transformation service is even available when it accepts the service request. The Service consumer may get some form of acknowledgement back from the service leading it to believe its request is being processed when in fact it cannot be completed. This would become much more complex in the real world if you layered protocols like WS-RX over top of the transactions.
Finally - As with OO programming and analysis, object should break large jobs down into several smaller tasks which can be completed and orchestrated at will. The third example above is akin to procedural web services. It also introduces dependencies between three components which may result in problems maintaining the system.
Conclusion:
A core tenet of SOA is managed transparency. Consumers should not make assumptions about what happens behind the service, nor should they demand to see details of such they do not really need.
So what do you think? I wm interested in hearing some counter points but please feel free to agree also.
Subscribe to:
Posts (Atom)