Sep 10 2012

Asp.net Mvc and the Nightmare of Dates and Time Zones

Category: Asp.net | MVCFrancesco @ 03:06

Who never suffered because of Time zones problems in his application? I am speaking of dates that returns changed to the server without being modified by the user! This is just one of the two nightmares connected to the dates handling in web applications, the other one being the different formats of dates in different cultures. I aready discussed the second nightmare in this post about globalization, so in this post I will focus just on how to handle properly Time zones in web applications.

Someone, might say: “I am not interested in this subject because in my applications both the browser and the server share the same Time zone!”. Unluckly this is not the case! If you don’t care of Time zone related issues you might have problems also when browser and the server share the same Time zone!!

The point is that all json serializers try to send the date to the client in UTC format(The 0 Time zone date), so they may transform the date by adding the Time zone offset. Now when the json date reaches the client a javascript date is created using the UTC format, but it shows up in the Time zone of the Borwser, so if both browser and server are on the same Time zone the dates appear to be exactly the same in both the browser and the server. For instance, say you have 11:00 time on the server that is in the +02 Time zone, this date will be converted into 09:00 before being serialized into json. When this date reaches the client the browser creates a 09:00 UTC date, but before showing it, or before returning the Hour it will add again the Time zone offset of the browser, so the date will show again as 11:00 in the browser if browser and server share the same Timzone! And what in case browser and server are in different Time zones? Simple: the browser will show the time of the server converted into the Time zone of the bowser.

That’s right! It is exactly what we want…so what is the problem? Why sometimes we receive the wrong date on the server?…Because when the client sends the time to the server it sends to the server the UTC time, that is 09:00 , but this time the de-serializer doesn't transform automatically the date in the local Time zone…so we receive 09:00…that appears to be the wrong hour…I said “appears” because it is not the wrong hour!

In order to understand why the above behaviour is correct, we need to know that in .Net DateTime structures have a property called Kind whose possible values are:

  • DateTimeKind.Unspecified
  • DateTimeKind.Utc
  • DateTimeKind.Local

All deserializers just return a date in Utc format with the Kind set to DateTimeKind.Utc simply because they dont know how the remainder of the system processes the dates, so they use the neutral Utc format.

So if we need the date in local time we just need to convert it by calling the ToLocalTime method of the DateTime structure!

So, it is quite simple…or…not? Unluckly this doesnt solve the Time zone nightmare! In fact as one might expect the DateTimeKind.Unspecified value creates problems, and unluckly most of data sources (such as databases) set the Kind property just to DateTimeKind.Unspecified .

So…what if the kind is set to DateTimeKind.Unspecified ? The serializer dont know if the Date Time is expressed in the local Time zone or in Utc, so some serializers assume it is already in Utc format and will not subtract the Time zone offset before sending it to the client while others will subtract the Time zone offset. In particular:

  • The JSON.NET serializer used as default by WebApi controllers assumes that the date is in Utc format and doesnt subtract the Time zone offset
  • The Microsoft JavaScriptSerializer used by the remainder of the Asp.net Mvc framework assumes the date is in the local Time zone format and do subtract the Time zone offset.

As a conclusion to avoid problems, please substitute all Datetimes with DateTimeKind.Unspecified with DateTimes with the right Kind before serializing them to json.

Unluckly this substitution cannot be done easily in case of WebApi action methods that return an IQueryable<T> and are decorated with the [Queryable] attribute. In fact the IQueryable is executed after the query contained in the Url is applied to it by the [Queryable] action filter, so the substitution can be done only after this event by another action filter!

In the Mvc Controls Toolkit we defined an action filter that do this job. Below an example of usage:

  1. [Queryable, DateConversionFilter(DateConversionFilterOperation.DeclareLocale)]
  2. public IQueryable<ToDoView> Get()
  3. {
  4.  
  5.     return ToDoViewModel.GetToDoQueryable();
  6. }

Obviously we can also use DateConversionFilterOperation.DeclareUTC if we know all dates are in the UTC format.

Finished?….Wait….Wait….There are two more issues to discuss to get rid of the Time zone nightmare.

  1. As we discussed before dates that are sent in json format to the browser are shown in the local Time zone of the client, but dates that are processed directly on the server and put into input html fields are shown in the Time zone of the server, since the conversion into a string is done on the server. Accordingly, it is not adviced to mix the two techniques to send dates to the browser if browser and server may be in different Time zones. However, also dates rendered on the server maybe expressed in the client Time zone if we detect someway the Time zone of the browser.
  2. Dates only fieds whose time part is not processed on the client may create problems because of the truncation of the time part if server and browser are in different Time zones.

To understand better the problem stated in point 2. let suppose we have a day stored in the server database, say, 2012-01-10. When this date is transferred into a DateTime it becomes: 2012-01-10 00:00:00. Now suppose the Time zone of the server is +02, when this DateTime is transformed into  Utc before the serialization its value becomes : 2012-01-09 22:00:00. Now if the browser Time zone is +01, this date shows up as 2012-01-09 23:00:00 into an input fields,…but since the time part is truncated…it becomes simply 2012-01-09 …that is 2012-01-09 00:00:00. Thus it is sent to the server as an Utc date whose value is 2012-01-08 23:00:00. The sever adds 2 hours to the above value to get the date in local time, yielding: 2012-01-09 01:00:00, that is stored again in the database as 2012-01-09  with an error of one day!

Adding an offset to avoid this problem doesnt solve since this will cause problems with other client Time zones. There are just two ways two solve the problem properly:

  1. Avoiding truncations. This means that the time part that is not shown to the user must be stored someway and added to whatever input day the user insert. In the Mvc Contols Toolkit if we use Client Blocks and we decorate the input field with the data-date-only=”true” Html5 attribute, this job is done automatically by the Client Blocks engine.
  2. Rounding the result at midnight of the nearest day after each conversion of the DateTime into a different format. It can be proved mathematically, that the above technique yields the right result. Now if the date is stored with a 0 time part, rounding must occurr just after the date is transformed into the local time of the client, on the client side, and after the date is received again on the server. The two roundings must be done in exactly the same way. In the Mvc Controls Toolkit we defined the following helper methods to perform easily this job:
    • On the client side: MvcControlsToolkit_DateTimeToDate(date).
    • On the server  side: ControllerUtilities.RoundDateTimeToDate(DateTime x), and ControllerUtilities.RoundDateTimeToDate(DateTime? x). There is also a mehod that rounds all dates contained in a ViewModel: ControllerUtilities.RoundDateTimesToDates<T>(T x).  The rounding can be performed also while transforming the date from Utc to local time: ControllerUtilities.DatesToLocale<T>(T x, bool round=false) that converts to local time and rounds all dates contained in a ViewModel.

 

Summing up, in order to handle poperly dates we need:

  1. To ensure all dates have the right Kind property attribute before serializing them, possibly with the help of an action filter.
  2. To transform all dates received by the server from their Utc format into the desired format.
  3. To deal adequately with dates only fields with the help of the data-date-only=”true” attribute and of the rounding technique.

Now we have…actually finished…examples showing the techniques discussed in this post are contained in the Mvc4 Client- Filtering -Paging -Sorting-updating and Advanced JSon Communication files of the download area of the Mvc Controls Toolkit.

 

                       That’s All

                                     Francesco

Tags: , , ,

Jul 3 2012

Mvc Controls Toolkit Support to Mvc4 WebApi 3: Retrieval Manager and other New Nice Features

Category: MVC | Asp.net | WebApiFrancesco @ 21:47

Mvc Controls Toolkit Support to Mvc4 WebApi

Mvc Controls Toolkit Support to Mvc4 WebApi 2: Handling Relations

In my previous posts about the Mvc4 WebApi support I described how the jQueryable and updatesManager Javascript classes may help us in all tasks related respectively with the retrieval of data from the server, and their updates. The jQueryable helps us in building the query to send to an Api Controller or to any oData source, either with a fluent LinQ like interface or by extracting the query information from the Mvc Controls Toolkit filtering, sorting, and paging controls. However, it doesn’t handle all aspects of the ClientViewModel update once we receive the data from the server. Moreover, there is also some query building job to be done “manually”.

In the 2.2 release of the Mvc Controls Toolkit we factored out all this “manual” jobs into the new mvcct.retrievalManager javascript class. Now all we need to do is just intercepting all events triggered by the Mvc Controls Toolkit filtering, sorting, and paging controls, and passing their data to the event method of the mvcct.retrievalManager instance we have created:

  1. $('#root').bind('queryChanged', function (e, data) {
  2.         ClientToDoView.retrieval.event(data);
  3.     });

Where the retrieval property of our client side ViewModel contains our instance of the mvcct.retrievalManager, and root is the id of a div containing all   filtering, sorting, and paging controls inside it. Since the filtering controls doesn’t trigger any event, we triiger the “filter” event with the refresh button of the forrm:

  1. <input id="Button1" type="button" data-bind='click: function(x, y){$(y.target).trigger("queryChanged", {type: "filter", filterPrefix: "ToDoFilter"});}' value="Refresh" />

The data object of the “queryChanged”  event triggered when the filtering criteria change must have a type of “filter”, and must contain the name of the server side ViewModel property that we use for fitering in its filterPrefix property:

  1. @{var hName = Html.DataFilterClauseFor(
  2.                   m => m.ToDoFilter, f => f.Name,
  3.                   "byNameFilter",
  4.                   MVCControlsToolkit.Linq.FilterCondition.StartsWith);}

As you can see from the code above this name is just ToDoFilter.

The instance of mvcct.retrievalManager can be created in the document ready event:

  1. $(document).ready(function () {
  2.     if (!window['ClientToDoView']) return;
  3.  
  4.     ClientToDoView.retrieval = mvcct.retrievalManager(query, ClientToDoView.DataPage.CurrPage, ClientToDoView.DataPage.TotalPages,
  5.     {
  6.         pageSize: 4,
  7.         entitiesContainer: ClientToDoView.DataPage.ToDoList,
  8.         updatesManager: ClientToDoView['updater'],
  9.         jFormsToClear: $('#mainForm'),
  10.         onError: function (args, x) {
  11.             var exception = $.parseJSON(x.responseText);
  12.             var message = mvcct.utils.isString(exception) ? exception : (exception.Message || "Internal Server Error");
  13.             alert("status code: " + x.status + "; " + message);
  14.         }
  15.     });
  16.  
  17.  
  18.     $('#root').bind('queryChanged', function (e, data) {
  19.         ClientToDoView.retrieval.event(data);
  20.     });
  21.     //populate initial results
  22.     ClientToDoView.DataPage.CurrPage(1);
  23.     ClientToDoView.retrieval.event({
  24.         type: "page",
  25.         page: 1
  26.     })
  27.     
  28. });

Where:

  • query, is the jQueryable object to be used to issue the queries.
  • The second argument is the observable that contains the current page number
  • The third argument is the observable that contains the total number of pages. It can be also null.
  • pageSize, contains the desired page size. This value can be changed at a later time by calling the changePageSize(newPageSize) method.
  • entityContainer, contains the observable where to put the array of objects returned by the server.
  • updatesManager, contains the updatesManager that handles the updates of the same array of objects, if any. If no update is needed, there is no need to pass this argument.
  • jFormsToClear, when specified is a jQuery object that contains some forms to clear from previous errors after having received new data from the server(once new data arrive, orld errors become obsolete).
  • onError, contains the function to execute in case of errors. It accepts 4 arguments: args, x1, x2, x3. args contains the data of the queryChanged event that triggered the request to the server, while x1, x2, x3 are the arguments passed to the error function of the ajax jQuery method.

There are are also other properties of the option object that we have not specified in this example:

  • onSuccess, if specified, overrides the standard model update behaviour of the retrievalManager. It accepts 4 arguments: args, x1, x2, x3. args contains the data of the queryChanged event that triggered the request to the server, while x1, x2, x3 are the arguments passed to the success function of the ajax jQuery method.
  • We can also keep the standard model update behaviour of the retrievalManager while simultaneously executing some custom code after the model has been updated, by specifying the onAfterSuccess function that accepts the same arguments of the onSuccess function. In this case we can specify also the dataTransform function that is passed all items returned by the server, as javascript objects before their properties are turned into ko observables. It is expected to apply a transformation to this data, and then return the array of transformed items.
  • onSubmit, if specified is executed before submitting the request to the server. It accepts two arguments: args, data. args give us the opportunity to cancel the request to the server, by setting args.cancel=true. data is the object we can provide in the data property of the option object.
  • immediateSubmit, has a default value of true. This means that each time a new event is passed to the retrievalManager a new request to the server is issued. If we set this property to false, the request to the server(with the query by processing  all events received so far) is issued manually by calling the submit() method of the retrievalManager.
  • resultsField and countField have respectively default values "Results" and "TotalCount". They are not used if the server returns just an array of objects, but they are used just if the server returns also the total count of the objects satisfying the query(information needed to improve the paging experience). In this case they specify the name of the properties containing the results and the total count. As default ApiControllers doesnt return the total count, but it is quite easy to write an action filter to add this value. Complete oData sources, such as Wcf oData web services return the total count if the query requires it. In such a case the default names suffices

The 2.2 release of the Mvc Controls toolkit improved also the updatesManage. Now there is no need to declare separately a child updatesManager. All information needed can be specified when adding it as a child to its father updatesManager. Below the updatesManagers of my previous posts Mvc Controls Toolkit Support to Mvc4 WebApi 2: Handling Relations rewritten with the new api:

  1. ClientToDoView.updater = mvcct.updatesManager(
  2.         '@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})',
  3.          ClientToDoView,
  4.          'DataPage.ToDoList',
  5.          'id', DestinationViewModel, "ToDoCS");

 

  1. ClientToDoView.childUpdater = ClientToDoView.updater.addChildUpdateManager({ expression: 'Tasks', external: 'FatherId' }, 'Key', "TaskCS");

As you can see, there is no need to create separately, the updatesManager that handles the children objects of our ToDo items. In fact a single instruction declares the the information needed to handle the one to many relation(‘Tasks’ and ‘FatherId’ are respectively the collection of the ToDo item containing the children objects, and the external key of the children objects), together with the key of the children entities (‘Key’) and the property of the destination model that will contains the updates of the children entities(‘TaskCS’) to send to the server. This is enough for the child updater to work properly

The  Mvc4 Client-Filtering -Paging-Sorting-updating file in the download area of the download area of the Mvc Controls toolkit contains both the new and old versions of the examples.

There are also other improvements that enhances the capability of the updatesManager to edit one to many relations. Namely:

  • The move method to move a child object from a father to another. This operation can be undone, as all other insert, update, and delete operations.
  • Now an updatesManager can be declared to be child of itself. This means that now the updatesManager is able to handle a one to may relation of an object set with itself. For instance we can handle a database table of tasks items where each task can be a subtask of another task in a recursive fashion.

 

See how the the controls of upcoming Data Moving plugin can take advantage of the new capabilities of the updatesManager to allow the user to edit one to many relations with the help of sophisticated TreeViews:

Below the updatesManager that handles the updates of the tasks TreeView in the video:

  1. HumanResourcesViewModel.TaskUpdater = mvcct.updatesManager(
  2.         '@Url.Action("TaskChanges", "TreeView")',
  3.          HumanResourcesViewModel,
  4.          'AllTasks',
  5.          'Id');

 

  1. HumanResourcesViewModel.TaskUpdater.addChildUpdateManager({ expression: 'Children', external: 'Father', updater: HumanResourcesViewModel.TaskUpdater });

As you can see the updatesManager is declared child of itself

That’s all for now!

                        Stay Tuned

                       Francesco

Tags: , , , , , , , , ,

Apr 11 2012

Mvc Controls Toolkit Support to Mvc4 WebApi 2: Handling Relations

Category: WebApi | MVC | Entity Framework | Asp.netFrancesco @ 21:51

In this post we will see how to handle one-to-many relations among entities with the advanced tools introduced in the last 2.1 release of the Mvc Controls Toolkit. We will give also some hints on how to handle some types of many-to-many relations that one might face in practical applications. I assume that everyone already read my previous post: Mvc Controls Toolkit Support to Mvc4 WebApi. The code of this example together with the code of my previous post is available in the  Mvc4 Client-Filtering -Paging-Sorting-updating file in the download area of the Mvc Controls Toolkit web site. Since, the update features we are going to discuss are not specific for ApiControllers but they are available also to standard Mvc 3 controllers, I provided also similar code examples that works with Mvc 3 controllers. They are in the AdvancedJSonCommunication file in the download area of the Mvc Controls Toolkit web site

As first step let modify our IQueryable to include also child entities of our ToDo items:

  1. return context.ToDo.Include("SubTasks").Select(item =>
  2.             new ToDoView()
  3.             {
  4.                 Name = item.Name,
  5.                 Description = item.Description,
  6.                 DueDate = item.DueDate,
  7.                 id = item.id,
  8.                 Tasks = item.SubTasks.Select(x => new SubTasksView { Name = x.Name, WorkingDays = x.WorkingDays, Key = x.code, FatherId = x.idTask })
  9.             });

We added a SubTasks table to our DB that is related in a one-to-many fashion to our ToDo table, and we are asking LinQ to add the children entities of each ToDo item through the Include clause.

We changed also the code of the WebApi method that exposes the IQueryable on the web:

  1. public IQueryable<ToDoView> Get()
  2. {
  3.     return new HttpSafeQuery<ToDoView>(ToDoViewModel.GetToDoQueryable(), true);
  4. }

We pass true as second argument of the constructor of our HttpSafeQuery. This way we instruct it to accept filtering criteria that are the logical and of clauses only. In fact, our application only needs such criteria, so by blocking all other kinds of queries we have a better protection against malicious users.

 

We show the children entities just in the detail window of our application:

Children

To achieve this result we can use just a simple client side  for statement:

  1. @{var h=item._foreach(m => m.Tasks, ExternalContainerType.tbody);}
  2. @h._begin()
  3. <tr>
  4.      <td>@h.TextBoxFor(m => m.Name)</td>
  5.      <td>@h.TypedTextBoxFor(m => m.WorkingDays, new { @class = "smallNumbers" })</td>
  6.      <td><input id="btnDetailDelete" type="button" value="Delete" data-bind='click: function(item){detailToDo.removeTask(item);}'/></td>
  7.      <td><input id="btnDetailUndo" type="button" value="Undo" data-bind='click: function(item){detailToDo.undoTask(item);}, enable: _inserted() || _modified()'/></td>
  8. </tr>
  9. @h._end()

 

The above code renders our children entities inside a table tbody(ExternalContainerType.tbody). The remainder of the table is written in Html.

Displaying the children entities was easy. The difficult part is handling their updates. The good news is that we don’t have code too much: the Mvc Controls Toolkit takes care of everything.

We need just to add a new ChangeSet property to the action method that receives the updates:

  1. public HttpResponseMessage<ApiServerErrors<int>> Post(UpdateViewModel model)

 

  1. public class UpdateViewModel
  2. {
  3.     public Updater<ToDoView, Int32> ToDoCS { get; set; }
  4.     public ChildUpdater<SubTasksView, Int32> TaskCS { get; set; }
  5. }

ChildUpdater is a subclass of the base Updater class we use to receive a change set that contains a property more. we will speak about it later onin this post. We are not forced to use the ChildUpdater and/or the Updater classes. we can use any class containing property to receive inserted, deleted,  modified…etc,  entities () by adequately declaring the names of such properties in our UpdateManagers.

We need also a another UpdateManager that takes care of the SubTasks entities and a destinationViewModel that is the  javascript equivalent of our UpdateViewModel. The destinationViewModel will be filled automatically by our UpdateManagers and submitted to the Post method of our controller.

We can declare the destinationViewModel as an empty object because the needed properties will be created automatically by the UpdateManagers that fill them:

  1. var DestinationViewModel = {};

You can find the above instruction in the EditDisplayToDo.js file while the definition of the UpdateManagers is contained in the IndexEdit.cshtml class to take advantage of the Url.RouteUrl method to compute the Url of the receiving action method:

  1. ClientToDoView.childUpdater = mvcct.updatesManager(
  2.     '@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})',
  3.      ClientToDoView,
  4.      'DataPage.TasksList',
  5.      'Key', DestinationViewModel, "TaskCS");
  6.  
  7. ClientToDoView.updater = mvcct.updatesManager(
  8.     '@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})',
  9.      ClientToDoView,
  10.      'DataPage.ToDoList',
  11.      'id', DestinationViewModel, "ToDoCS");

Each UpdateManager just specifies:

  1. The Url where to submit the change sets
  2. The source Client ViewModel that contains the entities
  3. A string expression that locates the property that contains the entities to handle within the source Client ViewModel.
  4. A string expression that identifies the principal key of each entity.
  5. The destination ViewModel
  6. A string expression that locates the change set within the destination ViewModel.

When the UpdateManagers compute the change sets they creates automatically the TaskCS and ToDoCS properties if they are not already defined in the destination ViewModel. Actually there is no DataPage.TasksList property in the source ViewModel since the SubTasks entities are contained in their father ToDo entities. The DataPage.TasksList is automaically created and filled with all the SubTasks entities just before computing the change sets.

 

Now we declare that the ClientToDoView.childUpdater works as a child of the main ClientToDoView.updater:

  1. ClientToDoView.updater.addChildUpdateManager({ expression: 'Tasks', external: 'FatherId', updater: ClientToDoView.childUpdater });

The object passed as argument of the call contains:

  1. expression: a string expressions locating the collection that contains the children entities within each ToDo item. In our case: Tasks.
  2. external: the external key of the child entities.
  3. updater: the updateManager to add as a child.

Newly created ToDo items need to be prepared and inserted with the Insert method:

  1. var item = {
  2.     DueDate: ko.observable(this.DueDate()),
  3.     Name: ko.observable(this.Name()),
  4.     Description: ko.observable(this.Description()),
  5.     id: ko.observable(null),
  6.     Tasks: ko.observableArray(this.Tasks())
  7. };
  8. ClientToDoView.updater.prepare(item, true); //newly created entity prepare it
  9. ClientToDoView.updater.inserted(ClientToDoView.DataPage.ToDoList, item);

Where the second argument of the prepare method ask to start changes tracking immediately.

While newly created SubTasks are added to the collection of their father entities with the addChild method of the father updateManager:

  1. detailToDo.createTask = function () {
  2.     var item = {
  3.         Name: ko.observable(''),
  4.         WorkingDays: ko.observable(0),
  5.         Key: ko.observable(null),
  6.         FatherId: ko.observable(null)
  7.     };
  8.     //newly created entity preparation is done when adding to father with addChild
  9.     ClientToDoView.updater.addChild(this, 'Tasks', item, true);
  10. };

 

Where:

  • this, denotes the father ToDo item
  • Tasks is the collection where to add the newly created item
  • item, is the newly created item
  • true, starts changes tracking immediately.

As already discussed in my previous post, deletes are performed by calling the deleted method:

 

  1. ClientToDoView.updater.deleted(ClientToDoView.DataPage.ToDoList, item);

 

  1. ClientToDoView.updater.deleted(detailToDo.Tasks, item);

As discussed in my previous post each time the user performs some modifications on an entity we must call the modified method that verifies if the entity actually changed and mark it as modified. For the ToDo items we do this in the save method of our detail window:

  1. detailToDo._save = function () {
  2.     var item = this.DetailOf();
  3.     if (!item) return true;
  4.     if (!$('#detailForm').validate().form()) return false;
  5.     mvcct.utils.restoreEntity(this, item, true);
  6.     ClientToDoView.updater.modified(item, true, true);
  7.     if ((!item._modified()) && (!item.tasksChanged()))
  8.         ClientToDoView.childUpdater.refreshErrors($('#mainForm'), null, item);
  9.     return true;
  10. };

For a description of the arguments of the modified method, please refer to my previous post.

For the SubTasks we have no Done button that is clicked after the user finished modifications, so we have to call automatically the modified function each time a property is modified. We can do this by attaching a function to each observable poperty of the SubTasks through the knockout subscribe method:

  1. ClientToDoView.childUpdater.options({
  2.     isoDate: true,
  3.     prepareCallback: function (item) {
  4.         var prev = false;
  5.         function subscription() {
  6.             ClientToDoView.childUpdater.modified(item, true, true);
  7.             if (prev && !item._modified())
  8.                 detailToDo.undoTask(item);
  9.             prev = item._modified();
  10.         };
  11.         item.Name.subscribe(subscription);
  12.         item.WorkingDays.subscribe(subscription)
  13.     }
  14. });

We have done this job in the prepareCallback function that its automatically called immediately after an entity is prepared. The preparedCallback can be declared in the updateManager options that are passed to the updateManager either as last argument of the constructor or through the options method.

We define a prepareCallback also for the ToDo entities:

  1. ClientToDoView.updater.options({
  2.     isoDate: true,
  3.     updateCallback: function (e, result, status) { alert("status: " + status); },
  4.     prepareCallback: function (item) { item.tasksChanged = ClientToDoView.updater.arrayChanged(item.Tasks); }
  5. });

It attaches a knockout computed to the taskChanged property of each ToDo item. This computed returns true if any child SubTask of the Tasks collection has been marked as modified. The knockout computed is returned by calling the arrayChanded method of the updateManager and passing it the collection we would like to check. Each time the state of a child entity changes the taskChanged property is recomputed automatically by the knockout engine. We can use the arraChanged property together with the standard _modified and _inserted properties, added to each entity, to enable the undo button if and only if there are changes to undo:

  1. <input type="button" value="Undo" data-bind='click: function(item){detailToDo.undo(item);}, enable: _inserted() || _modified() || tasksChanged()'/>

The Undo method calls the reset method of the main update manager that undoes all changes performed to the ToDo item and to all its children entities:

  1. detailToDo.undo = function (item) {
  2.     this.resetIfSelected(item);
  3.     ClientToDoView.updater.reset(item, $('#mainForm'));
  4. };

The reset method accepts a form as second argument, because when changes are undone possible server errors associted to thel undone entities are cancelled.

That is enough! Now when we submit the ClientToDoView.updater main updateManager the receiving controller receives the change set of the SubTasks entities, together with the change set of the ToDo entities automatically. Moreover, possible updateCallback and updatingCallback associated with the children updateManagers are called as appropriate. Summing up all we need to do is just:

  1. ClientToDoView.updater.update($('#mainForm'));

Once the change sets reach the controller they can be processed separately as needed by the business layer…However, we are committed to return to the client the newly created keys of the Inserted records. Thus our business methods must return them in the same order of the inserted entiies received in the change set.

There is another complication: inserted records that are children of inserted records! This records need the external key from their father entities before being processed and stored in the DB. Therefore we process the fater entity to get their principal keys:

  1. ToDoKeys = ToDoViewModel.UpdatePage(model.ToDoCS.Inserted, model.ToDoCS.Modified, model.ToDoCS.Deleted);

and then we pass them to possible children entities that might need them by calling the ImportExternals method of the ChildUpdater class:

  1. //imports the external keys of the newly created father entities into their children
  2. if (model.TaskCS != null) model.TaskCS.ImportExternals(ToDoKeys, m => m.FatherId)

Now we can process the children entities:

  1. if (model != null && model.TaskCS != null)
  2. {
  3.     TaskKeys=ToDoViewModel.UpdatePageTasks(model.TaskCS.Inserted, model.TaskCS.Modified, model.TaskCS.Deleted);
  4. }

everything is enclosed into a single transaction:

  1. try
  2. {
  3.     using (var t=new TransactionScope())
  4.     {
  5.         if (model != null && model.ToDoCS != null)
  6.         {
  7.             ToDoKeys = ToDoViewModel.UpdatePage(model.ToDoCS.Inserted, model.ToDoCS.Modified, model.ToDoCS.Deleted);
  8.  
  9.             //imports the external keys of the newly created father entities into their children
  10.             if (model.TaskCS != null) model.TaskCS.ImportExternals(ToDoKeys, m => m.FatherId);
  11.             //here the same for other children collections
  12.         }
  13.         if (model != null && model.TaskCS != null)
  14.         {
  15.             TaskKeys=ToDoViewModel.UpdatePageTasks(model.TaskCS.Inserted, model.TaskCS.Modified, model.TaskCS.Deleted);
  16.         }
  17.         t.Complete();
  18.     }
  19.     
  20. }
  21. catch (Exception ex)
  22. {
  23.     ModelState.AddModelError("", ex.Message);
  24.     return
  25.         new HttpResponseMessage<ApiServerErrors<int>>(
  26.             new ApiServerErrors<int>(ModelState, new ApiKeyInfos<int>[0]), System.Net.HttpStatusCode.InternalServerError);
  27. }

All new keys must be returned to the client to keep the client synchronized with the server:

  1. // if keys have different simple types such as one is int, and one other is string, use ApiServerErrors<object>
  2. return new ApiServerErrors<int>(ModelState,
  3.     new ApiKeyInfos<int>[] {
  4.         new ApiKeyInfos<int>{destinationExpression="ToDoCS", keys=ToDoKeys},
  5.         new ApiKeyInfos<int>{destinationExpression="TaskCS", keys=TaskKeys}
  6.     }).Wrap();

The keys handling algorithm complicates the interaction between the presentation layer and the business layer. This “too strong” interaction between layers maybe avoided by using Guids as keys. In fact, in this case all keys can be computed by the controller, so the whole key handling can be performed by the controller avoiding the need of  a “too strong“ cooperations between the presentation and the business layer. It is worth to point out that, due to security problems, Guids cannot be computed in the browser, so we are forced to compute them in the controller and to return them to the client.

For a better user experience the 2.1 release of the Mvc Controls Toolkit introduces two new features to handle the errors returned by the server:

  1. Automatic errors delete
  2. Errors bubbling

When an entity is undone, someway, either by pressing the undo button or by undoing manually all modifications, or by doing everytning else that might set its _modified status to false, all errors associated with it are deleted by the list of all errors returned by the server. This way, the next time the refreshErrors method of the updateManager is called they are deleted from the UI. The reset method of the updateManager calls automatically the refreshErrors method after having performed the undo, passing it the form that it receives as argument. This way all errors are cancelled from that form immediately. If the same error is displayed in several forms we need to call manually the refreshErrors method for all other forms:

  1. detailToDo.undoTask = function (item) {
  2.     ClientToDoView.childUpdater.reset(item, $('#mainForm'), this.Tasks);
  3.     ClientToDoView.childUpdater.refreshErrors($('#detailForm'));
  4. };

The first argument of the reset method is the item to undo, the second argument the form to refresh, and finally the third argument is the collection the item belongs to. The third argument is needed only in case of children entities, because not-children entities are not contained in any father collection.

Due to error bubbling, errors in children entities affect also the UI of their father entities, thus we have to refresh two forms: one is refreshed automatically by the reset method, and the other is refreshed by calling manually the refreshErrors method.

The errors associated with an entity can be deleted also manually by calling the refreshErrors method and passing it: the form as first argument, a null second argument, and the item we would like to delete the errors of as third argument. By passing null as second argument we ask the refreshErrors method to use the errors returned by the last call to the server.

Errors of any child entity are bubbled up to the property collection of the father entity that the children entity belongs to. However, just the fact that there was at least one error in the collection is bubbled up, not all error messages. This means that a ValidationMessageFor helper for the collection property will display just the error message passed as second argument. In our example we placed a ValidationMessageFor with an * near the edit button:

  1. <td>
  2.     <input type="button" value="Edit" data-bind='click: function(item){detailToDo.edit(item);}'/>
  3.     @item.ValidationMessageFor(m => m.Tasks, "*")
  4. </td>

Let see how errors work in practice. As first step let uncomment the error messages in the action method:

  1. //uncomment to experiment server side error handling
  2. //ModelState.AddModelError("ToDoCS.Modified[0].Name", "Fake error");
  3. //ModelState.AddModelError("TaskCS.Modified[0].Name", "Fake error1");

Then let start the application and let modify a ToDo item together with one of its children entities:

ErrorBubblingStart

Then let it Submit All Changes:

ErrorBubbligEnd

As you can see the error in the child entity has been bubbled up near to the edit button.

Now if we click the edit button, in the detail window we will just see the error of the child entity.

Why? Simple, the ToDo entity is copied into another ToDo entity that is bound to the detail window with the mvcct.utils.restoreEntity(x, y, visitRelations), while its children entities are used directly without creating a copy of them since setting visitRelations to true causes all children entity to be copied by reference into a new observableArray without being cloned. Since the errors are tied to the entity they belong to, just the errors of the child entity is shown. This was decided by design, to avoid duplication of the errors in the UI:

errorBubblingDetail

If we would like to show the ToDo entitiy errors also in the detail window we need to istantiate a template on the original ToDo entity by means of the _with helper instead of copying its data into another object that is already bound to UI elements.

Now let click the undo button of the children entity, or better simply delete the “modified” word we added previously. “Fake error 1” disappears from both the main window and from the detail window:

ErrorDeleteDetail

Now, if we click the undo button of the father entity, also “Fake error” completely disappears from the UI!

 

What if we have a many-to-many relation?

In the case of the one-to-many relation we just query our controller to get the “father entities” we need to process. As result we get the “father entities” and their related children entities “attached” to them. In the case, of a many-to-many relation we have no easy way to decide which entities of the second endpoint of the relation to move to the client. For this reason typically many-to-many relations cannot be processed in a batch fashion but they require a continuous interaction with the server. However, there is a common pattern that allows their batch processing, namely when one of the two endpoints of the relation is small enough to be transferred completely to the client.

This happens, for instance, when we have a collection containing all US States that we can select and attach to other entities. In other words, when one of the two related collections is used just for selection with a dropdown, or a multiselect or with some more complex UI. In this case the second collection of entities is used for display only. So we basically process just the collection R that represents the relation by means key pairs .

We can easily handle the situation depicted above with the same techniques we used for the one-to-many relations as follows:

  1. We move completely the second entity set to the client.
  2. We move to the client the entities of the first entity set we would like to process
  3. We move the entitis of R related to the entities at point 2 to the client. This can be accomplished in two ways
    • Add the entities of R as children of the entities of the first entity set they are related to,
    • Move all the entities of R we need as a separate collection and then connect them to the related entities of the first set of entities on the client with the help of the addRelated method of their updateManager (see my previous post).
  4. Define the updateManager of R as child of the updateManager of the first entity set.
  5. Connect the enties of R with the entities of the second entity set with the help of the addRelated method.
  6. To Add a new element to R, add it to the adequate entity of the first entity set, then set the external key of the second entity set, and add to it the pointer to the related entity of the second entity set.
  7. Process the entities of the first entity set + the entities of R with substantially the same techniques we used for the one-to-many relations, by accessing the pointer from each entity in R to the related entity of the second entity set when we need to display the related data.

That’s all for now! In a short time the team of the jsAction project will give us a very easy way to use the updateManager, by inspecting all controllers and providing automatically the right instances of the updateManager already configured to work with each specific controller.

                                               Stay Tuned

                                               Francesco

Tags: , , , , , , ,

Apr 3 2012

Mvc Controls Toolkit Support to Mvc4 WebApi

Category: Asp.net | MVC | WCF | WebApiFrancesco @ 03:57

The new Mvc 4 ApiController, and in particular the possibility it offers to expose an IQueryable on the web simplifies the exchange of data between server and an Html page, a lot.

Now it is possible to pass a dynamically created query from the client to the server with a simple GET operation. We need just to fill the $filter, $orderby, $skip, and $top query string parameters according to the oData protocol.

The upshot.js library may help us on the client side to write the code to interact with the server. Upshot does actually all the job, and make the developer interacts with the data as they were on the client, hiding all hard plumbing, needed to retrieve them, and to update them on the server.

However, the price we pay, to take advantage of all upshot.js features is that we can’t have substantially any business layer on the server side, whose role becomes a kind of “Web Database”.

What if we need a “robust” Business Layer, that manipulates data coming from several collections and/or objects before passing a transaction to the database?

At moment upshot.js contains DataProviders for interacting with:

  1. DbDataController. That is a particular ApiController specialized in interfacing DataBase data.
  2. oData sources. A complete oData source can be implemented with Wcf Data Services. In fact both a generic ApiController exposing an IQueryable, and the specialized DbDataController doesn’t implement the full oData protocol, but just support the $filter, $orderby, $skip, and $top operations, that are the more useful operations in business applications.
  3. Wcf Ria Services. The famous Ria services we use also with Siverlight.

One might think to build a custom DataProvider, and pass it to Upshot.js….However, a similar solution would never give us the “kind of customization” we need in this case. The point is that we can no more look at the server like a kind of “Web DB”, so we cannot use upshot and the DbDataController at all, and we need to proceed manually by building our custom jQuery ajax calls to generic WebApi methods.

However, there is some “hard plumbing job” we can factor out also in this case, namely:

  1. Building oData queries in a more friendly fashion with a LinQ like fluent syntax
  2. Changes tracking on the data. Recording the initial state of our data, so that we can undo all changes performed on them
  3. Computation of the Change Set to send to the server. This way we can save bandwidth by sending to the server just the modifications performed on data, instead of a whole collection of data
  4. Dispatching the errors returned by the sever in the right place in the User Interface, that is near the UI that displays the “wrong” inputs. Possibly, in the right labels rendered by the ValidationMessageFor helpers, and in the ValidationSummary.
  5. Since Json is not able to encode references, that is several properties pointing to the same instance of an object, but just a flat tree of objects, wee need someway to re-compute references links on the client after different collections containing related data have been sent by the server. Some tool that might help us in performing this job on the client might be very useful

Finally if we would like to take full advantage of WebApi also in business applications that manipulate big quantity of data we need to protect someway the IQueryables that we expose on the web to prevent denial of service attacks.

We can decorate our methods with the ResultLimitAttribute to limit the maximum number of items retrieved by the DB, However this is not enough, since a malicious user might try a denial of service attack by requiring a filtering operation on a column without indices of a big DB table. In this case also if no result is found at all the Data Base might waste a lot of time to discover it.

In the new Mvc4 beta compatible version of the Mvc Controls Toolkit we included tools that face all the problems listed above, namely:

  1. The javascript mvcct.Queryable abstract class allow us to build complex queries with a LinQ like fluent syntax. Moreover, it is able to import sorting, filtering, and paging information from all Mvc Controls Toolkit controls. There are three implementation of this abstract class:
    • The mvcct.oData.Queryable to query remote datasources that implements (also partially) the oData protocol. This is the right tool to use to query WebApi that expose an IQueryable
    • The mvcct.upshotQueryable build queries to be executed by the upsjhot.js library.
    • The localQueryable is able to query data that are already on the client in javascript format. It is useful when we need to work offline
  2. The mvcct.updateManager is able to track the changes of a collection of data, to compute the associated Change Set and to post it to a generic WebApi or standard Mvc Action Method. It is also able to transfer the data to the server with a standard browser submit (it creates dynamically a form that it then submits). Moreover it wait for the result from the server, and process possible errors returned by the server, by dispatching them in the right places of the UI. The developer may also provide two optional callbacks to be called respectively in case of success and in case of error. The error processing can be skipped by setting a property contained the callback arguments.
    Several mvcct.updateManager may cooperate to submit several change sets to the server with an unique post. In this case each mvcct.updateManager prepares the Change Set for the collection it takes care of, and put it in the destination ViewModel. Then, just one of them, the master, posts this ViewModel and wait for the server result . Once the result arrives it notifies all other mvcct.updateManager so that each of them can process its errors, and its optional callbacks.
  3. The addRelated method of the mvcct.updateManager computes automatically all mutual references existing between the items of two related collections by using key and external key information.
  4. The SafeQuery and HttpSafequery implementations of the IQueryable interface work as wrappers to protect any IQueryable from unwished queries. The allowed sorting and filering operations are specified through the CanSort Mvc Controls Toolkit attribute. In case a forbidden operation is detected the SafeQuery throws the adequate subclass of the ForbiddenException, while the HttlpSafeQuery that is specialized for the Web throws an HttpResponseException to set the right Http status code.

But let see how all this works in practice with a simple example. The example I use can be downloaded from the Mvc ControlsToolkit download area here: Mvc4 Client- Filtering -Paging -Sorting-updating. The file AdvancedJSonCommunication contains similar examples that works with standard Mvc3 contollers.

As first step let ‘s run the example to see what happens. We can select two pages in the menu: Index, and IndexEdit.The first one is a query and display only example, while the second one gives also the possibility to update the entities. Let start with the Index page.

ToDoDisplay

There is a pager, we can sort by clicking on the columns, and we can filter data by selecting several filtering options simultaneously and then by clicking Refresh

Let select filter by name, and let choose the Contains operator.

ToDoFiltering

Looking for a string contained in any position of a column is a very inefficient type of search if we have not defined a special index called full text index on that column. So it might be used for a denial of service attack against our website. However, when we  hit refresh we get:

ToDoError

A 403 Http status code: FORBIDDEN! The message explains us that the Contains operation is not among the operation allowed on the colum name.

Let see what happened behind the curtain:

  1. public class ToDoController : ApiController
  2.     {
  3.         // GET /api/todo
  4.         public IQueryable<ToDoView> Get()
  5.         {
  6.             return new HttpSafeQuery<ToDoView>(ToDoViewModel.GetToDoQueryable());
  7.         }

We wrapped our IQueryable within the HttpSafeQuery IQueryable that rejected the Contains clause of our filter.

The data annotations on the ToDoView class are provided through a MetaDataType(to understand why I did ths way, give a look here):

  1. [MetadataType(typeof(MetaToDo))]
  2.     public partial class ToDoView
  3.     {
  4.         public int? id { get; set; }
  5.        
  6.         public string Name { get; set; }
  7.        
  8.         public string Description { set; get; }
  9.  
  10.        
  11.         public DateTime DueDate {set;get;}
  12.  
  13.     }

So Let go to the MetaToDo class:

  1. public class MetaToDo
  2.     {
  3.         
  4.         [Required, CanSort, Display(Name="Name")]
  5.         public object Name { get; set; }
  6.         [Required, Display(ShortName = "Description")]
  7.         public object Description { get; set; }
  8.         [CanSort, Display(ShortName = "Due Date"), Format(DataFormatString="{0:D}")]
  9.         public object DueDate { get; set; }
  10.     }

The Name property is decorated with the CanSortAttribute, but since no argument is passed to specify wich filtering options are allowed, the default settings are taken…and the default setting doesn’t allow the dangerous Contains operator.

Now, let give a look to the way the query is built. We created the mvcct.oDataQueryable  object in the View to take advantage of the @Url.RouteUrl method to conpute the Url of our WebApi:

  1. <script type="text/javascript">
  2.     var query = mvcct.oDataQueryable('@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})', mvcct.$$.and);
  3.     query.importSortingControl('@Html.PrefixedId(m => m.ToDoOrder)');//import default sorting (necessary for paging
  4.  
  5. </script>

The second parameter of the constructor is the logical operator to be used to combine all filter conditions. We might have specified also mvcct.$$.or. There is also a third boolean parameter that we have not used in this example, negate. If true the whole filter is negated.

The first operation we do immediately is importing the initial sorting specified in the control that is bound to the ToDoOrder ViewModel property. It was put there by the action method that took care of creating the Index View:

  1. public const int PageDim=5;//in actual application this should be put in a config file
  2.         public ActionResult Index()
  3.         {
  4.             //define default ordering. It is necessary for paging.
  5.             List<KeyValuePair<LambdaExpression, OrderType>> order = new List<KeyValuePair<LambdaExpression,OrderType>>();
  6.             Expression<Func<ToDoView, DateTime>> defaultOrder = m => m.DueDate;
  7.             order.Add(new KeyValuePair<LambdaExpression, OrderType>(defaultOrder, OrderType.Descending));
  8.  
  9.             ToDoViewModel result = new ToDoViewModel()
  10.                 {
  11.                     DataPage = new ToDoPage {
  12.                         ToDoList=new List<ToDoView>()},
  13.                     ToDoOrder=order
  14.  
  15.                 };
  16.             return this.ClientBlockView(result, "ClientToDoView");
  17.         }

As you can see this action method just creates the substantially empty ViewModel that will be used as page ViewModel by the knockout.js library

The adviced way to use knockout.js with the Mvc Controls Toolkit is through the Client Blocks feature. For more information on the sorting Mvc Controls toolkit controls give a look here.

The remaining code used to build the query is in the WebApiRetrieve.js file:

  1. ClientToDoView.Refresh = function (type, page, sorting, prevPage) {
  2.         if (!type) {
  3.             type = 'filter'; page = 1;
  4.             ClientToDoView.DataPage.CurrPage(1);
  5.             //filter changed, we need reset pager to the first page
  6.         }
  7.         query.setPaging(page, pageSize);
  8.         //sorting changed
  9.         //we don't need to reset pager to the first page, sorting control do it automatically
  10.         if (type == 'sort') query.importSorting(sorting);
  11.         else if (type == 'filter') {
  12.             query.resetFilter()
  13.             .importClauses('ToDoFilter');
  14.         }
  15.         query.execute(function (x) {.......

The Refresh method of the ViewModel is called either when the filter, or the sorting or the page is changed, so the first thing it does is “understanding” what changed. Then it imports into the query the changed information. The pager and the sorting controls triggers an event containing the new page and/or sorting information when something changes, so we need just to register an event to handle sorting and paging:

  1. $('#root').bind('queryChanged', function (e, data) {
  2.         ClientToDoView.Refresh(data.type, data.page, data['sortString'], data['previousPage']);
  3.     });

sortString, and previousPage are put within [‘’] because the sortString property is provided just for the sorting changed event, while the previousPage is provided just for the page changed event, so both properties may be undefined.

There is no event that says us the when filtering condition changes, so the best we can do is binding the refresh method of the ViewModel to the Refresh button through a knockout click binding:

  1. <input id="Button1" type="button" data-bind='click: function(x){x.Refresh();}' value="Refresh" />

The mvcct.Queryable.importClauses imports all filtering conditions by inspecting directly all filtering controls that are bound to the  ToDoFilter ViewModel property. For an introduction to the MvcControls Toolkit Filtering controls give a look here.

The execute method issues the query to the WebApi controller, and receives the results through a callback, whose code is not shown.

Basically the callback inspects the format of the object that is returned to see if the total count of all entities is available (this information may be used to improve the paging experience). WebApi methods doesn’t supplies this information and return just a collection of entities, but full implementation of the oData protocol like the Wcf Data services may return it. Then they add observables and observableArray, as needed by calling the ko.mapping.fromJS function of the knockout mapping plugin:

  1. var newEntities = ko.mapping.fromJS(x.results)();

and put them into the ViewModel:

  1. ClientToDoView.DataPage.ToDoList(newEntities);

The total count of the entiies is updated either with the exact information supplied from the server, if available, or with a smart guess:

  1. ClientToDoView.DataPage.TotalPages(totPages);

The pager is bound to the DataPage.TotalPages ViewModel property, and immediately takes advantage of this new information. In case of errors the pager is reset to its previous page:

  1. ClientToDoView.DataPage.CurrPage(prevPage);

The pager is bound to this ViewModel property, and it updates its state immediately. The ko bindings of the pager are automatically created by the pager Html helper.

In case we would like to use the upshot.js library to query our controller, we can stll import paging, filtering, and sorting infos from the Mvc Controls toolkit controls by using the upshotQueryable in place of the the oDataQueryable, since both objects share the same public interface.

The constructor in this case is: mvcct.upshotQueryable(dataSource, fop, options, negate). Where dataSource is an upshot datasource, fop and negate are completely ignored since upshot allows just the logical and of all filter conditions. Finally the option object contains only the includeTotalCount property whose default value is true;

we can also build our query manually with the help of the whole fluent public interface exposed by all mvcct.Queryable objects:

  • setPaging(page, pageSize); Specifies current page and page size. we already used it in our example.
  • addSort(field, desc, enabled): if enabled is true adds a new orderby clause to the order being built. field is the name (actually a string expression) of the property, and the boolean desc specifies if the order is descending.
  • resetSorting():clears all previously added sorting clauses, so that a completely new sorting can be built.
  • addCondition(operator, value1, value2, enabled): if enabled is true a new filtering condition is added to the filter being built. All filter conditions are combined with the currently active logical operator. The initial logical operator is specified in the constructor of the Queryable. A different logical operator can be specified each time we build a complex condition made of several sub-conditions with the help of the open and close methods (see later), value1 is the property to be constrained by the condition,  value2 the second argument of the comparison , and operator is the condition operator  that is one of:
    • mvcct.$$.eq, mvcct.$$.ne, mvcct.$$.gt, mvcct.$$.ge, mvcct.$$.lt, mvcct.$$.le;
    • mvcct.$$.startswith, mvcct.$$.endswith
    • mvcct.$$.substringofInv; that is the Contains operator.
    • mvcct.$$.substringof; that requires the property value is a substring of the value2 argument
  • resetFilter(): clears all previously added filter clauses, so that a completely new filter can be built.
  • open(logicalOperator, enabled, negate): if enabled is true, it starts a complex condition made of several sub-conditions that will be combined with logicalOperator. If negate is true the whole condition will be negated. Its effect is opening a parenthesys in the filter expression being built.
  • close(enabled): close a previously opened parenthesys. enabled must match the enabled of the matching open otherwise the parenthesys of the resulting expression will be unbalanced.
  • importSorting, importSortingControl, and importClauses are the Mvc Controls Toolkit controls importing methods we have already used in our example. There is also an importPager(pagerId, pageSize) method that imports the paging data contained in the pager whose id is pagerId. we have not used it in the example because we took the all page informations from the event triggered by the pager.
  • get(): returns an object whose nature depends on the implementation of the Queryable: the oDataQueryable returns the complete url to be passed to the server, the localQueryable returns a function that once executed performs all operations specified in the query, and the upshotQueryable returns the upshot DataSource.
  • execute(callback): executes the query and passes the result returned by the server as argument to the callback. The upshotQueryable ignores the callback because the only operation performed is the refresh of the DataSource: this is enough to dispatch the server results in the right place.
  • getState() and setState(x) respectively returns an object that encodes the full state of the Queryable, and set the state of the Queryable by using a previously saved state. They are useful to handle the Back and Forward buttons of the browsers to navigate the history of all query passed to the server.

It is time to understand how to handles data updates with the updateManager class. In our example the creation of the  instance of the updateManager that takes care of our ToDo items is in the IndexEdit view:

  1. ClientToDoView.updater = mvcct.updatesManager(
  2.         '@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})',
  3.          ClientToDoView,
  4.          'DataPage.ToDoList',
  5.          'id', null, null, {isoDate:true, updateCallback: function (e, result, status) { alert("status: " + status); } });

The first argument just computes the Url of our WebApi, the second argument is the javascript variable that contains our page ViewModel, the third argument is a string expression that locates the collection we would like to process. The fourth argument is another string expression that locates the “key” of our items. It doesn’t need to correspond to the actual DB key of the item, but it is just the field that we want our updateManager send to the server when an item is deleted.

The further two arguments that we set to null are respectively, the destination ViewModel and a string expression that locates the place in the destination ViewModel where to put the change set once it has been computed. Since both fields are null there is no destination ViewModel, and the change set will be sent to the server, as it is , without inserting it into another container. The destination ViewModel is useful if we need to send simultaneously several change sets to the server.

The change set is an object with just 3 properties: one containing the array of all modified items, another one containing the array of all newly inserted items, and the last property contains the array of the “keys” of all deleted items. The names of these properties can be gonfigured in the option argument of the constructor of the updateManager (the last argument). The default option object is:

  1. {
  2.     updater: {u: "Modified", i: "Inserted", d: "Deleted"},
  3.     isoDate:false,
  4.     updateCallback: function(e, result, status){},
  5.     updatingCallback: function (changes, modelToPost, expr){return changes;}
  6. }

The first property contains the names of the three fields of the change set object.

The second property specifies if the date format to use when sending the change set to the server is the ISO format or the \/Date(….)\/ format. When sending data to a standard Mvc controller we have to set isoDate to false, while in our case we can’t use the default since WebApi controllers need the ISO format.

The updateCallback is called just before processing possible errors returned by the server, and it receives:

  • e. an object with format:
    { setErrors: true, model: sourceViewModel, expression: sourceExpression, key: keyExpression, success: !result.errors }
    If in our callback we set the setErrors property to false, error processing is not carried out, and we have to process them with our custom logics.
  • result. The result returned by the server, that must  be an object containing an error property with the list of all errors and an optional insertedKeys property with the keys created dynamically when inserting new records in the DB. Both the errors and the keys received from the server are processed automatically. That is, the errors are dispatched in the right places of the UI, and the keys to all newly created items that need them.The result object can contain also further properties that can be processed in a custom way  in the updateCallback.
  • status: the Http status of the server response. In our example we specify a callback that just shows the status with an alert window.

The updatingCallback is invoked just before sending all data to the server. If this callback returns a false, null, or undefined value the update is aborted. The updatingCallback receives three arguments:

  • changes: a boolean that informs us if any change was detected
  • modelToPost: the model that the updateManager is going to post
  • expr: a string expression that locates the just computed changes set within the above model.

Let give a look to our receiving action method:

  1. HttpResponseMessage<ApiServerErrors<int>> Post(Updater<ToDoView, int> model)

The Updater generic class that is defined in the MVCControlsToolkit.Controller namespace is just the server side equivalent of our javascript change set class. It is just an helper class since we can use any class to receive the change set. It is enough to specify the right property names in the updateManager option object.

We can use also a complex object containing several change set objects that matchs our javascript destinationViewModel, as parameter  to deal with several simultaneous collection updates. Moreover, we are not limited to using webApi methods but we can use also standard controller action methods.

The full code of the controller method  is:

  1. public HttpResponseMessage<ApiServerErrors<int>> Post(Updater<ToDoView, int> model)
  2. {
  3.     //uncomment to experiment server side error handling
  4.     //ModelState.AddModelError("Modified[0].Name", "Fake error");
  5.     int[] insertedKeys;
  6.     if (ModelState.IsValid)
  7.     {
  8.         insertedKeys=ToDoViewModel.UpdatePage(model.Inserted, model.Modified, model.Deleted);
  9.         
  10.     }
  11.     else
  12.     {
  13.         insertedKeys = new int[0];
  14.     }
  15.     return new ApiServerErrors<int>(ModelState, insertedKeys).Wrap();
  16. }

If the ModelState is valid we call a business layer method, otherwise we abort business computation and return just the list of errors. It is enough to invoke the constructor of the server side equivalent of our javascript result  object passing it the ModelState, and an array with the keys of all newly created items, to create our result object filled with all errors and/or keys. The call to the Wrap method embeds this object into a response containing an appropriate Http status code. In case we have a complex destinationModel with several change sets, we can use a different overload of the constructor that accepts several arrays of keys, each with a string expressoin that specifies the change set of the destinationViewModel it refers to, Namely:

  1. public ApiServerErrors(ModelStateDictionary origin, ApiKeyInfos<T>[] insertedKeys, string prefix = null)

where:

  1. [DataContract]
  2. public class ApiKeyInfos<T>
  3. {
  4.     [DataMember]
  5.     public string destinationExpression { get; set; }
  6.     [DataMember]
  7.     public T[] keys { get; set; }
  8. }

There is also a not generic version of the ApiServerErrors class that we can use when we don’t need to send keys to the client.

The ApiServerErrors class is contained in the MVCControlsToolkit.Controller namespace together with the analogous ServerErrors class that we can use with standard controllers action methods.

Let uncomment the line that creates a fake error and let see how errors are dispatched to the client:

ToDoEditing

Let modify a couple items, and let add a new one, then let hit submitAllChanges.

since the user input was wrong we get a 400 status code (BAD REQUEST):

error400

Then since the property we added the error is: Modified[0].Name, the error is dispatched to the ValidationMessageFor label next to the name of the first modified item:

ValidationError

Since we had no luck with our changes we can decide to undo everything by hitting the Undo All Changes button: Everything is cleared…changes and errors.

How does Undo and Change Set computation work? How to enable them in our applications?

The first step is to prepare all items that we receive from the server:

  1. var newEntities = ko.mapping.fromJS(x.results)();
  2. if (ClientToDoView['updater']) {//if entities may be modified and sent back to the server, prepare them
  3.     ClientToDoView.updater.prepare(newEntities, true);
  4. }

or that we create on the client:

  1. detailToDo.saveAsNew = function () {
  2.     if (!$('#detailForm').validate().form()) return;
  3.     var item = ko.mapping.fromJS({
  4.         DueDate: this.DueDate(),
  5.         Name: this.Name(),
  6.         Description: this.Description(),
  7.         id: null
  8.     });
  9.     ClientToDoView.updater.prepare(item, true);//newly created entity prepare it
  10.     ClientToDoView.updater.inserted(ClientToDoView.DataPage.ToDoList, item);
  11.     this.reset();
  12. };

Then, we must call the inserted method to insert them in our collection as shown above,  the deleted method to delete them:

  1. detailToDo.remove = function (item) {
  2.     this.resetIfSelected(item);
  3.     ClientToDoView.updater.deleted(ClientToDoView.DataPage.ToDoList, item);
  4. };

and the modified method each time the user manipulate them someway:

  1. detailToDo.save = function () {
  2.     var item = this.DetailOf();
  3.     if (!item) return;
  4.     if (!$('#detailForm').validate().form()) return;
  5.     mvcct.utils.restoreEntity(this, item);
  6.     ClientToDoView.updater.modified(item, true, true);
  7.     this.reset();
  8. };

The second and third argument of the modified method say respectively, to prepare the entity if it is not yet prepared(actually we don’t need it since we know entities have been already prepared), and to do an immediate verification of all changes. Thus all properties of our entity are compared with their old values to verify if an actual change occurred, and only if the entity actually changed it is marked as modified. If the third argument of the modified method is false, our entity is marked as modified without performing any immediate verification; the verification is deferred till the time the updateManager compute the change set.

The mvcct.utils.restoreEntity is an utility method of the Mvc Controls Toolkit  that copies an object into another object having its same structure, properly handlig all ko observables. As a default nested objects are visited and their properties copied but arrays are not. However, if its second optional argument is set to true, references to arrays are copied too.

Preparing a property add it two observable properties: _inserted and _modified, that say us the state of the entity. They are used by the routines that compute the change set, but we can use them also to improve the user experience. In our case we bound them to the enabled status of the undo button:

  1. <input type="button" value="Undo" data-bind='click: function(item){detailToDo.undo(item);}, enable: _inserted() || _modified()'/>

If in the call to the prepare method we pass true as second argument(as we have done), changes tracking is turned on, and the entity will remember its initial state. If changes tracking is off no undo is possible, and an entity is marked modified when we call the modified method without performing any check.

Below how to undo a single entity:

  1. detailToDo.undo = function (item) {
  2.     this.resetIfSelected(item);
  3.     ClientToDoView.updater.reset(item);
  4. };

it is enough to call the reset method of the updateManager. The updateManager has also a resetAll method that restore the initial state of the whole collection:

  1. ClientToDoView.undoAll = function () {
  2.     ClientToDoView.updater.resetAll($('#mainForm'));
  3.  
  4. };

It accepts, as argument, a jQuery object containing a form, that it uses to clear all errors. The state of the errors returned by the server can be also cleared manually each time we need by calling the clearErrors(jForm) method of the updateManager. In our example we do it when we perform a new query to the server:

  1. if (ClientToDoView['updater']) {//clear previous errors, since entities shown on the screen will change
  2.     ClientToDoView.updater.clearErrors($('#mainForm'));
  3. }

we can require manually also the dispatching of errors by calling the refreshErrors(jForm, errorState) where errorState is an object with the same format as the result returned by the server. If errorState is null the last errors returned by the server will be used.

The change set can be submitted to the server either by calling the update(jForm) method:

  1. ClientToDoView.save = function () {
  2.     ClientToDoView.updater.update($('#mainForm'));
  3. };

or by calling the submit(jForm) method, in which case a not-ajax normal browser submit is triggered. The form submitted is dynamically created and contains just the destinationViewModel data.Examples showing the use of the submit method and the use of the updateManager with standard controllers action methods are contained in the file Advanced JSon Communication in the download area of the Mvc Controls Toolkit

Both the update and the submit methods have a second argument: isDependent. It is used when submitting simultaneously multiple change sets to the server into an unique destinationViewModem. When isDependent is set to true the updateManager just compute the change set and insert it in the right place in the destinationViewModel without performing any operation on the server. The operation on the server is performed by an unique updateManager, called with isDependent set to false, that acts as Master. When we call the update method the Master dispatches automatically the results returned by the server to the Slave updateManagers, thus causing they update their, entities, their errors, and call their updateCallBack. The array of all Slaves to notify is passed as third argument of the update method.

The addRelated method of the updateManager can help us in handling multiple related collection on entities:

  1. addRelated(collectionExpression, entities, entitiesExternalExpression, inverseCollectionExpression, overrideKeyExpression)

It computes all mutual references existing between the items of two related collections by using key and external key information:

  • entities is the array(or ko observable array) with the collection we would like to relate with the collection handled by the current instance of the updateManager.
  • collectionExpression is a string expression specifying the array contained in each entity of the collection handled by the current instance of the updateManager,  where the pointers to the related entities will be pushed. If the array doesn’t exists it is created, it it already exists it is not cleared.
  • entitiesExternalexpression is a string expression that locates the external key within each entity of the entities collection.
  • inverseCollectionExpression is a string expression specifying the array contained in each entity of the entities collection where the pointers to the related entities will be pushed. If the array doesn’t exists it is created, if it already exists it is not cleared.
  • overrideKeyexpression, if provided, is used in place of the key defined in the current instance of the updateManager.

In a short time I will write a more detailed blog post on how to handle related collections.

All examples on the js Queryable and on the updateManager are contained in the files Mvc4 Client- Filtering -Paging -Sorting-updating and Advanced JSon Communication in the download area of the Mvc Controls Toolkit

In a short time the team of the jsAction project will give us a very easy way to use the updateManager, by inspecting all controllers and providing automatically the right instances of the updateManager already configured to work with each specific controller. For the moment enjoy the post of my Friend Vincenzo of the jsAction team.

Tags: , , , , , , ,