PREFACE
 
 It seems that the word "AJAX" has been flooded everywhere these days when people 
 are talking about web development. People use "AJAX" to label a much bigger territory, 
 Remote Scripting. AJAX is only formally supported by major browsers for a short period but Remote Scripting has 
 been used for a long long time (See one concrete example). If you think AJAX is as "simple" as 
       
         - create a XMLHttpRequest Object
         - make a call with callback handler
         - handle server response in the handler

 you are wrong. AJAX, no, Remote Scripting, means not only being able to make requests to server and update your
 pages partially without reloading the entire pages but also introduces a totally different manner/pattern of communication 
 to the server. The intention is "minimizing loading entire page" to save communication bandwidth, reduce server load,
 and optimize visual effect. On the other hand, Remote Scripting breaks some traditional "rules" that browser 
 originally introduced and people have got along with such as going Backward and Forward in web surfing history, 
 client-state preservation.

 The goal of this paper is starting at what AJAX has and what are shorted in "AJAX" to bring up most major 
 issues and their solutions/approaches on Remote Scripting. For each issue, illustrate them at two levels:

   1). A variety of solutions (either by Mark Qian or others) on remote scripting will be listed. 
   2). For each solution, a variety of live implementation will be shown(either by Mark Qian or others).
 
 
 
A. What features does AJAX have?
      AJAX, Asynchronous JavaScript and XML, is just a feature, relatively "newly" 
      being available in browsers which implemented it differently. So let's see 
      what this browser feature offers.     

 a.Communicate with server Asynchronously
   
Forces
Show details:
   

     The major technology AJAX relies on, in term of communication, is XMLHttpRequest (XHR). XMLHttpRequest 
     is mostly used to "silently" update a part or entire screen in a web application without reloading 
     web entire pages. I will illistrate a variety of solutions on "Communicate with server Asynchronously".
     For each solution, I will list different implementations.
       
     I will also show how to use a single "wrapper", Mark's Communicator, to wrap a variety of 
     implementation details through all the solutions.

   
   
Solutions
Show details:
   
           
AJAX
   

  XMLHttpRequest is used in this approach but there may be many low level details such as
  XMLHttpRequest pooling, progress indication and so on. One way to make life easier is to
  wrap them in a js object and even make transport (in term of DOJO) transparent to users.
  

  Here are a variety of implementation:

     1. Mark's implementation using Mark's Communicator for "an autocomplete component". 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo, Codes and Design.

     2. Mark's implementation using Mark's Communicator to build a on-demand js tree. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo, Codes and Design.
 
     3. Mark's implementation using Mark's Communicator to build a on-demand scrollable table. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.

      4. Mark's implementation using Mark's Communicator to build an AutoSuggest Text Box. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.

             
Dynamic hidden iFrame

Show details


  Dynamic hidden iFrame is used in this approach but unlike AJAX, we need to save the reference
  to handlers globally so that we can retreave them when server response return. One way to achieve
  this is to  wrap the implmentation details in a js object, say Communicator and have a "Mananger" 
  sites globally managing Communicators. I will show a variety of ways and how to use them below.
   
            
  Here are a variety of implementation:

     1. Mark's implementation using Mark's Communicator  for "an autocomplete component". 

         In this example, the iframe will be create dynamically and wrapped as a "Communicator" 
         so that the details of communication management such as pooling, progress indicator is transparent to 
         programmers.

         Demo, Codes and Design.

     2. Mark's implementation using Mark's Communicator to build a on-demand js tree. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo, Codes and Design.

     3. Mark's implementation using Mark's Communicator to build a on-demand scrollable table. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.

     4. Mark's implementation using Mark's Communicator to build an AutoSuggest Text Box. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.

             
Dynamic Scripting (cross-domain)

  Dynamic Scripting is used in this approach. Similar to iframes, we need to save the reference
  to handlers globally so that we can retreave them when server response return. One way to achieve
  this is to  wrap the implmentation details in a js object, say Communicator and have a "Mananger" 
  sites globally managing Communicators. I will show a variety of ways and how to use them below.
             
Show details
              
             
  Here are a variety of implementation:

     1. Mark's implementation using Mark's Communicator for "an autocomplete component". 

         In this example, the script section for communication will be create dynamically and wrapped as a "Communicator" 
         so that the details of communication management such as pooling, progress indicator is transparent to 
         programmers.

         Demo, Codes and Design.
         
     2. Mark's implementation using Mark's Communicator to build a on-demand js tree. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo, Codes and Design.
         
     3. Mark's implementation using Mark's Communicator to build a on-demand scrollable table. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.

     4. Mark's implementation using Mark's Communicator to build an AutoSuggest Text Box. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.


             
Hidden Frame

  The reason I am show this "old" way here is that it is the most compatible way 
  when something wrong with iframe such as it is too costful (in term of memory usage)
  using iframe some broswers. You can turn off iframe, XMLHttpRequest, Cookies, but you
  can not turn off static frames unless frames are not supported.

  In this example, hidden frame is wrapped as a "Communicator" so that the details 
  of communication management such as progress indicator is transparent to 
  programmers.

  Here are a variety of implementation:

     1. Mark's implementation using Mark's Communicator for "an autocomplete component". 

     Demo, Codes and Design.
         
     2. Mark's implementation using Mark's Communicator to build a on-demand js tree. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo, Codes and Design.

     3. Mark's implementation using Mark's Communicator to build a on-demand scrollable table. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.
                                    
     4. Mark's implementation using Mark's Communicator to build an AutoSuggest Text Box. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.

      
         
Image/Cookies
  
  The trick is using src of image to send the request and use cookies to send 
  and retreave respond. The limitation of this approach is that it relies on
  browser cookies. 

  Another usage of image trick is that in case you don't need the server respond (that is, you
  only want to send server a message to update some state at the server side but no need to update any state
  at the client side), you can use the image src to send your request.
  
Show details
 
  Here are a variety of implementation:

     Brent's RSLite is a typical implementation of this approach:
     "RSLite is an extremely lightweight implementation of remote scripting which uses cookies. 
      It is very widely browser-compatible (Opera!) but limited to single calls and small 
      amounts of data... " by Brent Ashley 

     1. Brent Ashley's implementation of "Get Server Var" using RSLite. 

        Demo and Codes.  

     2. Mark's implementation using Mark's Communicator to build a on-demand js tree. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.
          
          Demo, Codes and Design.
          

     3. Mark's implementation using Mark's Communicator to build a on-demand scrallable table. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.
                            
     4. Mark's implementation of "an autocomplete component" using Mark's Communicator with RSLite. 
               See how can I wrap RSLite within my Communicator to make it transparent to programmers.

     Demo and Codes.  
     
     5. Mark's implementation using Mark's Communicator to build an AutoSuggest Text Box. 

          In this example, XMLHttpRequest is wrapped as a "Communicator" so that the details 
          of communication management such as pooling, progress indicator is transparent to 
          programmers.

          Demo.
                           
           
  
           
Show details
            
   
   
 b. More "AJAX has" features will be added here ...

 
B. What problems does AJAX have?

      AJAX is actually a "subset" of "Remote scripting". Personally, I think it is not the best approach 
      (Why? See my explanation in "What is wrong with those most popular web application frameworks?") 
      amoung most of approches in "Remote scripting" territory. So it is why I use it as the starting 
      point to illustrate a variety of issues we need to face when remote scripting. 
      
      Let's take a look at the issues.  

 a. AJAX can't be used crossing domain
   
The Problem
Show details:
   
             XmlHttpRequest objects are limited by "the same origin" security policy of browsers.
             Under the policy, objects and pages can not access data from different server. Actually,
             this problem is not only AJAX's. It exists, "by default", on most remote scripting
             approaches.


             There are two demos for this policy:

               1). Client side communication. See the Demo showing the problem. 

               2). Client/Server communication. See the Demo showing the problem. 

             To workaround, people have been having a variety of approaches/solutions.
     
   
   
Solutions
Show details:
   

           
             
Dynamic scripting

   One approach is to make use of an exception of the "same origin" rule. The rule thinks
   "Any script directly included on a page, is assumed to come from the same domain as the page 
   it is included in". This approach is more feasible since there is not suffix constrain (see
   the "Changing document.domain" below)

     1. Mark's implementation allowing you contain content loaded from any 
               domain in a single page. 
     
      See the Demo in a non-frame page (div is used to divide areas for 
             content loaded from different domains)  here.

      See the Demo in a page with iframe (iframe is used to divide areas for 
             content loaded from different domains) here.
             
      See the Demo in Mark's On-demand Tree where the tree tries to update 
             its content from a different domain:  there are a variety of approaches 
             but on Dynamic Scripting works.            
             
     
     2. Articles about Dynamic scripting.

              - "AJAX with dynamic SCRIPT tags - revised" by Darryl Lyons' Blog

              - "Cross-domain Partial Page Updates with JavaScript" by Auke van Slooten

             
Changing document.domain

   Another one is trying to make browsers think two frames are loaded from the 
   "same origin".

   JavaScript provides the document.domain property allowing cooperating sites to 
   remove this restriction for their particular Web sites, i.e., pages which 
   specifically set the document.domain property can override (to an extent) 
   the Same-Origin policy to allow scripts to read and write values across 
   frames as if both pages did come from the same domain.
   
   Limitation:

     - You only set document.domain to a suffix of the original domain the page is loaded from.
     - Once document.domain is used in one page it needs to be set in all other pages
     - Some XmlHttpRequest in some browser such as Mozilla doesn’t acknowledge the 
       document.domain setting. So there is no way to communicate between different servers 
       directly through it.

   Abe Fettig introduced a his attempts on approach.
                   
   Demo and Codes
             
             
Proxy script in CPAINT
                   
   Codes

             
mod_rewrite (Available in Apache 1.2 and later)
                   
   API at Apache and Sample from premshree.livejournal.com
             
 Dynamic Script Tag & JSON

Codes from Jason Levitt.
             
Server-side reverse proxying
              
             
Bypassing XmlHttpRequest entirely
                    
             
More approaches to be added...
                                     
           
    
           
           Show details
           



 b. AJAX has problems with the "back" buttons and bookmark of browsers
   
The Problem
Show details:
   
   
       The reason I described AJAX as a "gun" is that it can be very helpful but could also be harmful
       if you use it "as is". When people talk about problems of AJAX, someone may immediately point out that
       "AJAX has Back Button problem". But the fact is that AJAX is also able to solve your Back Button
       problem.
       
       There are two kinds of "Back Button" problems caused by: 1).lost history or 2). "garbage" history.
       I will show you how AJAX cause the first type of problem but is able to solve the second one.
       
       

       
       Let's take a look at the first type of Back Button problem caused by lost history.
       
       In traditional web pages, what users see is what they can bookmark and go forward and backward.
       That is, every request users make back to the server is recorded in the browser's history.
       With AJAX, views in pages may be changed without reloading the pages and these changes are
       never recorded in some browser's history. The problem is that users can neither bookmark some 
       "views" they want to come back next time nor going back to a "previous view" (the fact 
       is sometimes the browser is completely moved out of the current site since the "previous view"
       of that site is never in the history).


       Here is how to see this kind of behaviour:
       
        i). Example A, a very simple one.
        
           If you try to bookmark this blue expanded view, you can't since the state recording the div expansion
           is store in javascript's memory not in the browser's history. Similarly, if click at "Back" or "Forward"
           buttons and come back here, you will see this div is collapsed: you won't see the original view with
           the blue div area expanded - the state is lost.  Browsers always starts each page with band new javascript
           memory when each page is loaded.
        
        ii). Example B

           1). navigate your browser to site say "microsoft.com"
           2). now go to a page with AJAX/Remote scripting, say "maps.google.com" (by typing
               "maps.google.com" into the browser's location field and hitting go).
           3). find your favor place, say "Mountain view, CA", by zooming and panning
           4). bookmark the current view by select Favorites|Add to Favorites (in IE)
           5). find another place in the map, say "Sunnyvale, CA", by zooming and panning
           6). now, click at your favor "Back" button trying to return to your favor place,
               "Mountain view, CA", in Google Maps. Suprisely , you will find that 
               you move out of Google on to the enemy's side, Microsoft.com.
           7). OK, you may think, well, I had bookmarked the place ("Mountain view, CA") in the map. 
               So you click at the bookmarked item in your favorite list but what you will see
               is the initial loading of Google Map the map of USA instead of "Mountain view, CA"
                         



       Let's now take a look at the second type of Back Button problem caused by "garbage" history.
       
       When you submit forms directly and return the same page back with red fields in case
       of validation failure, the Back button walks you through all the "garbage pages" instead of
       the real previous page.  See details in my article "Building an AJAX Application (3): 
       how to submit a form 'silently'?"
       
             Problem live code show
       
             Solution live code show
       
       We will see how AJAX (or other means of Remote Scripting) can be used to solve the problem.
       
               
       To workaround, people have been having a variety of approaches/solutions.
     
     
     Show details
     
   
   
Solutions
Show details:
   
          
           

             
Record Ajax actions and add them into the browser history for the first type of Back Button problem

        1. Mark's implementation using RSH. 
             The open source Really Simply History framework (RSH) is a typical approach of this kind.
 
              Demo and Codes.    
              
          
Mark silent form submission to solve the second type of Back Button problem
               
        1. Mark's implementation using Mark's Communictor.
                      
              Demo and Codes.
              
            




 c. AJAX does not preserve client-side page states
   
The Problem
Show details:
   
       Note: this problem is not "AJAX only". It may exist in any page keeping states in 
             javascript namespace but AJAX application is more "intend to" do that. And this problem is
             actually an abstract of the "Back" button problem.

       One nature of browsers is preserving page states in DOM components (in form) in their history
       so that you can easily bookmark pages and use "Back" and "Forward" buttons.       

       Say, before you complete your email editing in Hotmail.com, you like to copy and paste some 
       info from a different site like Google.com and then return the Hotmail.com using "Back" button 
       to complete your email edition. This is fine with a "traditional" pages like the "new mail" 
       page at hotmail.com. 

       But the way AJAX application works on many pages is "only load the page once". A lot of state 
       info is preserved in javascript memory. And they may have some initialization function to setup 
       the page only once since it assumes the page is only loaded once and any further communication 
       will be carried out "silently" by XMLHttpRequest object. The problem is when users click at 
       "Back" button to return to the provious page they lose the state since the javascript namespace 
       is "refreshed" (gone when you switch away from the site and reloaded when reenter the site) and 
       initialization codes are called to clean the value in DOM.

       Here is how you can try this kind of behaviour:
            
           1). go "map.google.com" (where AJAX/remote scripting is used)
           2). find whatever place you want to go (zooming and panning), say, your lovely hometown "Mountain view, CA",
           3). type the enemy's site, "microsoft.com", in to the address field of your browser and go
               to "microsoft.com"
           4). click at your favor "Back" button 
           5). Ooh ooh: you moved (the map is initialized to the original position): "Mountain view, CA" in gone in the
               view of Google map.


               
       To workaround, people have been having a variety of approaches/solutions.
     
     
     Show details
     
   
   
Solutions
Show details:
   
   
           

             
Similar to the approach above (making up browser history): write states into DOM so that browser will add them into history
         Let's try the experiment above again in a different way: 
         instead of finding your favor place by zooming and panning, use the "Search Maps" field.

           1). go "map.google.com" (where AJAX/remote scripting is used)
           2). find whatever place you want to go , say, your lovely hometown "Mountain view, CA", 
               by typing "Mountain view, CA" into the location field and click at "Search Maps" button.
           3). type the enemy's site, "microsoft.com", in to the address field of your browser and go
               to "microsoft.com"
           4). click at your favor "Back" button 
           5). Yes, you come back to "Mountain view, CA" in the map           

        Why is the state preserved now? The reason is that the state (the location "Mountain view, CA")
        is stored in one of the DOM component the location text field (in a form) while the states are kept in javascript
        objects when you change states by panning and zooming. That is, states resulted by panning and zooming
        are stored in javascript namespace instead of DOM in Google Maps so they never appear in the browser's history.
        (Of course, this is the intention of Google Map: never bother the browser history with tiny operations like
         panning and zooming).

        So you can obviously understand why people come out with this approach: don't (just) keep your state in javascript
        objects but (also) write them into hidden DOM components like iframes so that browser will remember them.

        Being "Remembered" in browser's history, on the other hand, may not always be a good thing. That may be a drawback 
        of this approach when you want to preserve the states but don't want them to appear in history.

        1. Brad Neuberg's approach is a typical one of this kind.

              He used iframe to record the state and recover it when reloading.
 
              Demo and Codes.

             
Keep state in cookies

        1. 
 
              Demo and Codes.


Show details


 d. AJAX may be over chating with server
   
The Problem
Show details:
   
       
   One of the major factors affecting scalability is high-frequency chatness between client and server.
   The intension of AJAX application is to "chat" more frequently with server. If this kind of communication
   is not carefully limited it may dramatically impact the scalability. Here are some examples:
       

  
On-demand table
           On-demand table loads new rows as users scroll to a new area. It minimize/eliminates the initial loading period but dramatically increate the chatness
           to server. 
       
       To workaround, people have been having a variety of approaches/solutions.
     
   
   
Solutions
Show details:
   
   

 e. AJAX is bypassing the core of browsers, the renderers
   
The Problem
Show details:
   
       It is in javascript's namespace (execution context of the callback handler function)
       when server response as "plain text" or dom is received. I said "plaint text" because
       even though you can include script in the return but it won't be executed automatically
       since the browser renderer is not involved.
       
   
   
Solutions
Show details:
   
           

             
AJAX
             You can parse the return "plaint text" to extract the script area (many people 
             place script in the header area) and eval it yourself. The problem with this 
             approach is that parsing and eval are not a fun job. You need to carefully escape
             special characters. And it becomes more complication when the script is not simple such
             as a lib and multiple closure functiions need to be invoked in certain sequence.
             In that case, your code logic may need to be duplicated: one for normally loading
             process by browser's renderers and the other for eval.
             
             Here are Demo and Codes.
             
Dynamic hidden iFrame
             This problem is not a problem to iframes since browser will render the return content
             and the script will be executed within the hidden iframe. And no extra logic is needed
             since you only use a single piece of code for both types of request.
             
             Here are Demo and Codes.
             
Hidden frame
             Similar to the iframe approach.
             
             Here are Demo and Codes.   
                                 
             
More approaches to be added...
                                     
           
  
           
           Show details
            
 
 f. AJAX is bypassing the browser's form submission functionality
   
The Problem
Show details:
   
       Similarly, it is in javascript's namespace when preparing parameters for submission.       
       
   
   
Solutions
Show details:
   
           

             
AJAX
             So you need to extract fields from the form to be submit and construct your
             parameter list before invoking XMLHttpRequest. You can always write a "cool"
             generic method to handle most forms but again, you duplicate the form code.
             
             Here are Demo and Codes.
             
Dynamic hidden iFrame
             This problem is not a problem to iframes since you never touch fields in the form.
             You only need to point the target of the form to the hidden iframe and
             nothing redundent.
             
             Here are Demo and Codes.
             
Hidden frame
             Similar to the iframe approach.
             
             Here are Demo and Codes.   
                                 
             
More approaches to be added...
                                     
           
   
           
           Show details
           
   
 g. AJAX could be turn off in some major browser like IE (6.x)

The Problem
Show details:
   

             AJAX is not a native component in some browsers and users may disable it. 
             For example, AJAX appears in IE as an ActiveX, an embedded object. Users or 
             security application can disable it in the "Script ActiveX controls marked 
             safe for scripting" from the "internet option" of control panel in Windows.
             Actually, a lots of companies will have ActiveX controls disabled and many
             PC protection programs also make turning off ActiveX at one of their security
             level.

             If your users are the "very end" users like shopping people using 
             IE to access your shopping carts, using AJAX is really gambling. 
             You are bidding on them with ActiveX on to see your AJAX pages.

             Even though IE 7.x makes XMLHttpRequest a native component but it can still
             be turn off.

   
   
Solutions
Show details:
   
           

             
AJAX
             You may want to detect the situation and work around.
             
             Here are Demo and Codes.
             
Dynamic hidden iFrame
             This problem is not a problem to iframes event though there is an item under 
             "Security Setting" named "Launching programs and files in an IFRAME". 
             It doesn't seem to work in IE6.
             
             Here are Demo and Codes.
             
Hidden frame
             This problem is not a problem to traditional frames at all.
             
             Here are Demo and Codes.   
                                 
             
More approaches to be added...
                                     
           
  
           
           Show details
           
          
                         
 h. More "AJAX doesn't have" features to be added...