Apr 11 2012

Mvc Controls Toolkit Support to Mvc4 WebApi 2: Handling Relations

Category: WebApi | MVC | Entity Framework | Asp.netFrancesco @ 21:51

In this post we will see how to handle one-to-many relations among entities with the advanced tools introduced in the last 2.1 release of the Mvc Controls Toolkit. We will give also some hints on how to handle some types of many-to-many relations that one might face in practical applications. I assume that everyone already read my previous post: Mvc Controls Toolkit Support to Mvc4 WebApi. The code of this example together with the code of my previous post is available in the  Mvc4 Client-Filtering -Paging-Sorting-updating file in the download area of the Mvc Controls Toolkit web site. Since, the update features we are going to discuss are not specific for ApiControllers but they are available also to standard Mvc 3 controllers, I provided also similar code examples that works with Mvc 3 controllers. They are in the AdvancedJSonCommunication file in the download area of the Mvc Controls Toolkit web site

As first step let modify our IQueryable to include also child entities of our ToDo items:

  1. return context.ToDo.Include("SubTasks").Select(item =>
  2.             new ToDoView()
  3.             {
  4.                 Name = item.Name,
  5.                 Description = item.Description,
  6.                 DueDate = item.DueDate,
  7.                 id = item.id,
  8.                 Tasks = item.SubTasks.Select(x => new SubTasksView { Name = x.Name, WorkingDays = x.WorkingDays, Key = x.code, FatherId = x.idTask })
  9.             });

We added a SubTasks table to our DB that is related in a one-to-many fashion to our ToDo table, and we are asking LinQ to add the children entities of each ToDo item through the Include clause.

We changed also the code of the WebApi method that exposes the IQueryable on the web:

  1. public IQueryable<ToDoView> Get()
  2. {
  3.     return new HttpSafeQuery<ToDoView>(ToDoViewModel.GetToDoQueryable(), true);
  4. }

We pass true as second argument of the constructor of our HttpSafeQuery. This way we instruct it to accept filtering criteria that are the logical and of clauses only. In fact, our application only needs such criteria, so by blocking all other kinds of queries we have a better protection against malicious users.

 

We show the children entities just in the detail window of our application:

Children

To achieve this result we can use just a simple client side  for statement:

  1. @{var h=item._foreach(m => m.Tasks, ExternalContainerType.tbody);}
  2. @h._begin()
  3. <tr>
  4.      <td>@h.TextBoxFor(m => m.Name)</td>
  5.      <td>@h.TypedTextBoxFor(m => m.WorkingDays, new { @class = "smallNumbers" })</td>
  6.      <td><input id="btnDetailDelete" type="button" value="Delete" data-bind='click: function(item){detailToDo.removeTask(item);}'/></td>
  7.      <td><input id="btnDetailUndo" type="button" value="Undo" data-bind='click: function(item){detailToDo.undoTask(item);}, enable: _inserted() || _modified()'/></td>
  8. </tr>
  9. @h._end()

 

The above code renders our children entities inside a table tbody(ExternalContainerType.tbody). The remainder of the table is written in Html.

Displaying the children entities was easy. The difficult part is handling their updates. The good news is that we don’t have code too much: the Mvc Controls Toolkit takes care of everything.

We need just to add a new ChangeSet property to the action method that receives the updates:

  1. public HttpResponseMessage<ApiServerErrors<int>> Post(UpdateViewModel model)

 

  1. public class UpdateViewModel
  2. {
  3.     public Updater<ToDoView, Int32> ToDoCS { get; set; }
  4.     public ChildUpdater<SubTasksView, Int32> TaskCS { get; set; }
  5. }

ChildUpdater is a subclass of the base Updater class we use to receive a change set that contains a property more. we will speak about it later onin this post. We are not forced to use the ChildUpdater and/or the Updater classes. we can use any class containing property to receive inserted, deleted,  modified…etc,  entities () by adequately declaring the names of such properties in our UpdateManagers.

We need also a another UpdateManager that takes care of the SubTasks entities and a destinationViewModel that is the  javascript equivalent of our UpdateViewModel. The destinationViewModel will be filled automatically by our UpdateManagers and submitted to the Post method of our controller.

We can declare the destinationViewModel as an empty object because the needed properties will be created automatically by the UpdateManagers that fill them:

  1. var DestinationViewModel = {};

You can find the above instruction in the EditDisplayToDo.js file while the definition of the UpdateManagers is contained in the IndexEdit.cshtml class to take advantage of the Url.RouteUrl method to compute the Url of the receiving action method:

  1. ClientToDoView.childUpdater = mvcct.updatesManager(
  2.     '@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})',
  3.      ClientToDoView,
  4.      'DataPage.TasksList',
  5.      'Key', DestinationViewModel, "TaskCS");
  6.  
  7. ClientToDoView.updater = mvcct.updatesManager(
  8.     '@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})',
  9.      ClientToDoView,
  10.      'DataPage.ToDoList',
  11.      'id', DestinationViewModel, "ToDoCS");

Each UpdateManager just specifies:

  1. The Url where to submit the change sets
  2. The source Client ViewModel that contains the entities
  3. A string expression that locates the property that contains the entities to handle within the source Client ViewModel.
  4. A string expression that identifies the principal key of each entity.
  5. The destination ViewModel
  6. A string expression that locates the change set within the destination ViewModel.

When the UpdateManagers compute the change sets they creates automatically the TaskCS and ToDoCS properties if they are not already defined in the destination ViewModel. Actually there is no DataPage.TasksList property in the source ViewModel since the SubTasks entities are contained in their father ToDo entities. The DataPage.TasksList is automaically created and filled with all the SubTasks entities just before computing the change sets.

 

Now we declare that the ClientToDoView.childUpdater works as a child of the main ClientToDoView.updater:

  1. ClientToDoView.updater.addChildUpdateManager({ expression: 'Tasks', external: 'FatherId', updater: ClientToDoView.childUpdater });

The object passed as argument of the call contains:

  1. expression: a string expressions locating the collection that contains the children entities within each ToDo item. In our case: Tasks.
  2. external: the external key of the child entities.
  3. updater: the updateManager to add as a child.

Newly created ToDo items need to be prepared and inserted with the Insert method:

  1. var item = {
  2.     DueDate: ko.observable(this.DueDate()),
  3.     Name: ko.observable(this.Name()),
  4.     Description: ko.observable(this.Description()),
  5.     id: ko.observable(null),
  6.     Tasks: ko.observableArray(this.Tasks())
  7. };
  8. ClientToDoView.updater.prepare(item, true); //newly created entity prepare it
  9. ClientToDoView.updater.inserted(ClientToDoView.DataPage.ToDoList, item);

Where the second argument of the prepare method ask to start changes tracking immediately.

While newly created SubTasks are added to the collection of their father entities with the addChild method of the father updateManager:

  1. detailToDo.createTask = function () {
  2.     var item = {
  3.         Name: ko.observable(''),
  4.         WorkingDays: ko.observable(0),
  5.         Key: ko.observable(null),
  6.         FatherId: ko.observable(null)
  7.     };
  8.     //newly created entity preparation is done when adding to father with addChild
  9.     ClientToDoView.updater.addChild(this, 'Tasks', item, true);
  10. };

 

Where:

  • this, denotes the father ToDo item
  • Tasks is the collection where to add the newly created item
  • item, is the newly created item
  • true, starts changes tracking immediately.

As already discussed in my previous post, deletes are performed by calling the deleted method:

 

  1. ClientToDoView.updater.deleted(ClientToDoView.DataPage.ToDoList, item);

 

  1. ClientToDoView.updater.deleted(detailToDo.Tasks, item);

As discussed in my previous post each time the user performs some modifications on an entity we must call the modified method that verifies if the entity actually changed and mark it as modified. For the ToDo items we do this in the save method of our detail window:

  1. detailToDo._save = function () {
  2.     var item = this.DetailOf();
  3.     if (!item) return true;
  4.     if (!$('#detailForm').validate().form()) return false;
  5.     mvcct.utils.restoreEntity(this, item, true);
  6.     ClientToDoView.updater.modified(item, true, true);
  7.     if ((!item._modified()) && (!item.tasksChanged()))
  8.         ClientToDoView.childUpdater.refreshErrors($('#mainForm'), null, item);
  9.     return true;
  10. };

For a description of the arguments of the modified method, please refer to my previous post.

For the SubTasks we have no Done button that is clicked after the user finished modifications, so we have to call automatically the modified function each time a property is modified. We can do this by attaching a function to each observable poperty of the SubTasks through the knockout subscribe method:

  1. ClientToDoView.childUpdater.options({
  2.     isoDate: true,
  3.     prepareCallback: function (item) {
  4.         var prev = false;
  5.         function subscription() {
  6.             ClientToDoView.childUpdater.modified(item, true, true);
  7.             if (prev && !item._modified())
  8.                 detailToDo.undoTask(item);
  9.             prev = item._modified();
  10.         };
  11.         item.Name.subscribe(subscription);
  12.         item.WorkingDays.subscribe(subscription)
  13.     }
  14. });

We have done this job in the prepareCallback function that its automatically called immediately after an entity is prepared. The preparedCallback can be declared in the updateManager options that are passed to the updateManager either as last argument of the constructor or through the options method.

We define a prepareCallback also for the ToDo entities:

  1. ClientToDoView.updater.options({
  2.     isoDate: true,
  3.     updateCallback: function (e, result, status) { alert("status: " + status); },
  4.     prepareCallback: function (item) { item.tasksChanged = ClientToDoView.updater.arrayChanged(item.Tasks); }
  5. });

It attaches a knockout computed to the taskChanged property of each ToDo item. This computed returns true if any child SubTask of the Tasks collection has been marked as modified. The knockout computed is returned by calling the arrayChanded method of the updateManager and passing it the collection we would like to check. Each time the state of a child entity changes the taskChanged property is recomputed automatically by the knockout engine. We can use the arraChanged property together with the standard _modified and _inserted properties, added to each entity, to enable the undo button if and only if there are changes to undo:

  1. <input type="button" value="Undo" data-bind='click: function(item){detailToDo.undo(item);}, enable: _inserted() || _modified() || tasksChanged()'/>

The Undo method calls the reset method of the main update manager that undoes all changes performed to the ToDo item and to all its children entities:

  1. detailToDo.undo = function (item) {
  2.     this.resetIfSelected(item);
  3.     ClientToDoView.updater.reset(item, $('#mainForm'));
  4. };

The reset method accepts a form as second argument, because when changes are undone possible server errors associted to thel undone entities are cancelled.

That is enough! Now when we submit the ClientToDoView.updater main updateManager the receiving controller receives the change set of the SubTasks entities, together with the change set of the ToDo entities automatically. Moreover, possible updateCallback and updatingCallback associated with the children updateManagers are called as appropriate. Summing up all we need to do is just:

  1. ClientToDoView.updater.update($('#mainForm'));

Once the change sets reach the controller they can be processed separately as needed by the business layer…However, we are committed to return to the client the newly created keys of the Inserted records. Thus our business methods must return them in the same order of the inserted entiies received in the change set.

There is another complication: inserted records that are children of inserted records! This records need the external key from their father entities before being processed and stored in the DB. Therefore we process the fater entity to get their principal keys:

  1. ToDoKeys = ToDoViewModel.UpdatePage(model.ToDoCS.Inserted, model.ToDoCS.Modified, model.ToDoCS.Deleted);

and then we pass them to possible children entities that might need them by calling the ImportExternals method of the ChildUpdater class:

  1. //imports the external keys of the newly created father entities into their children
  2. if (model.TaskCS != null) model.TaskCS.ImportExternals(ToDoKeys, m => m.FatherId)

Now we can process the children entities:

  1. if (model != null && model.TaskCS != null)
  2. {
  3.     TaskKeys=ToDoViewModel.UpdatePageTasks(model.TaskCS.Inserted, model.TaskCS.Modified, model.TaskCS.Deleted);
  4. }

everything is enclosed into a single transaction:

  1. try
  2. {
  3.     using (var t=new TransactionScope())
  4.     {
  5.         if (model != null && model.ToDoCS != null)
  6.         {
  7.             ToDoKeys = ToDoViewModel.UpdatePage(model.ToDoCS.Inserted, model.ToDoCS.Modified, model.ToDoCS.Deleted);
  8.  
  9.             //imports the external keys of the newly created father entities into their children
  10.             if (model.TaskCS != null) model.TaskCS.ImportExternals(ToDoKeys, m => m.FatherId);
  11.             //here the same for other children collections
  12.         }
  13.         if (model != null && model.TaskCS != null)
  14.         {
  15.             TaskKeys=ToDoViewModel.UpdatePageTasks(model.TaskCS.Inserted, model.TaskCS.Modified, model.TaskCS.Deleted);
  16.         }
  17.         t.Complete();
  18.     }
  19.     
  20. }
  21. catch (Exception ex)
  22. {
  23.     ModelState.AddModelError("", ex.Message);
  24.     return
  25.         new HttpResponseMessage<ApiServerErrors<int>>(
  26.             new ApiServerErrors<int>(ModelState, new ApiKeyInfos<int>[0]), System.Net.HttpStatusCode.InternalServerError);
  27. }

All new keys must be returned to the client to keep the client synchronized with the server:

  1. // if keys have different simple types such as one is int, and one other is string, use ApiServerErrors<object>
  2. return new ApiServerErrors<int>(ModelState,
  3.     new ApiKeyInfos<int>[] {
  4.         new ApiKeyInfos<int>{destinationExpression="ToDoCS", keys=ToDoKeys},
  5.         new ApiKeyInfos<int>{destinationExpression="TaskCS", keys=TaskKeys}
  6.     }).Wrap();

The keys handling algorithm complicates the interaction between the presentation layer and the business layer. This “too strong” interaction between layers maybe avoided by using Guids as keys. In fact, in this case all keys can be computed by the controller, so the whole key handling can be performed by the controller avoiding the need of  a “too strong“ cooperations between the presentation and the business layer. It is worth to point out that, due to security problems, Guids cannot be computed in the browser, so we are forced to compute them in the controller and to return them to the client.

For a better user experience the 2.1 release of the Mvc Controls Toolkit introduces two new features to handle the errors returned by the server:

  1. Automatic errors delete
  2. Errors bubbling

When an entity is undone, someway, either by pressing the undo button or by undoing manually all modifications, or by doing everytning else that might set its _modified status to false, all errors associated with it are deleted by the list of all errors returned by the server. This way, the next time the refreshErrors method of the updateManager is called they are deleted from the UI. The reset method of the updateManager calls automatically the refreshErrors method after having performed the undo, passing it the form that it receives as argument. This way all errors are cancelled from that form immediately. If the same error is displayed in several forms we need to call manually the refreshErrors method for all other forms:

  1. detailToDo.undoTask = function (item) {
  2.     ClientToDoView.childUpdater.reset(item, $('#mainForm'), this.Tasks);
  3.     ClientToDoView.childUpdater.refreshErrors($('#detailForm'));
  4. };

The first argument of the reset method is the item to undo, the second argument the form to refresh, and finally the third argument is the collection the item belongs to. The third argument is needed only in case of children entities, because not-children entities are not contained in any father collection.

Due to error bubbling, errors in children entities affect also the UI of their father entities, thus we have to refresh two forms: one is refreshed automatically by the reset method, and the other is refreshed by calling manually the refreshErrors method.

The errors associated with an entity can be deleted also manually by calling the refreshErrors method and passing it: the form as first argument, a null second argument, and the item we would like to delete the errors of as third argument. By passing null as second argument we ask the refreshErrors method to use the errors returned by the last call to the server.

Errors of any child entity are bubbled up to the property collection of the father entity that the children entity belongs to. However, just the fact that there was at least one error in the collection is bubbled up, not all error messages. This means that a ValidationMessageFor helper for the collection property will display just the error message passed as second argument. In our example we placed a ValidationMessageFor with an * near the edit button:

  1. <td>
  2.     <input type="button" value="Edit" data-bind='click: function(item){detailToDo.edit(item);}'/>
  3.     @item.ValidationMessageFor(m => m.Tasks, "*")
  4. </td>

Let see how errors work in practice. As first step let uncomment the error messages in the action method:

  1. //uncomment to experiment server side error handling
  2. //ModelState.AddModelError("ToDoCS.Modified[0].Name", "Fake error");
  3. //ModelState.AddModelError("TaskCS.Modified[0].Name", "Fake error1");

Then let start the application and let modify a ToDo item together with one of its children entities:

ErrorBubblingStart

Then let it Submit All Changes:

ErrorBubbligEnd

As you can see the error in the child entity has been bubbled up near to the edit button.

Now if we click the edit button, in the detail window we will just see the error of the child entity.

Why? Simple, the ToDo entity is copied into another ToDo entity that is bound to the detail window with the mvcct.utils.restoreEntity(x, y, visitRelations), while its children entities are used directly without creating a copy of them since setting visitRelations to true causes all children entity to be copied by reference into a new observableArray without being cloned. Since the errors are tied to the entity they belong to, just the errors of the child entity is shown. This was decided by design, to avoid duplication of the errors in the UI:

errorBubblingDetail

If we would like to show the ToDo entitiy errors also in the detail window we need to istantiate a template on the original ToDo entity by means of the _with helper instead of copying its data into another object that is already bound to UI elements.

Now let click the undo button of the children entity, or better simply delete the “modified” word we added previously. “Fake error 1” disappears from both the main window and from the detail window:

ErrorDeleteDetail

Now, if we click the undo button of the father entity, also “Fake error” completely disappears from the UI!

 

What if we have a many-to-many relation?

In the case of the one-to-many relation we just query our controller to get the “father entities” we need to process. As result we get the “father entities” and their related children entities “attached” to them. In the case, of a many-to-many relation we have no easy way to decide which entities of the second endpoint of the relation to move to the client. For this reason typically many-to-many relations cannot be processed in a batch fashion but they require a continuous interaction with the server. However, there is a common pattern that allows their batch processing, namely when one of the two endpoints of the relation is small enough to be transferred completely to the client.

This happens, for instance, when we have a collection containing all US States that we can select and attach to other entities. In other words, when one of the two related collections is used just for selection with a dropdown, or a multiselect or with some more complex UI. In this case the second collection of entities is used for display only. So we basically process just the collection R that represents the relation by means key pairs .

We can easily handle the situation depicted above with the same techniques we used for the one-to-many relations as follows:

  1. We move completely the second entity set to the client.
  2. We move to the client the entities of the first entity set we would like to process
  3. We move the entitis of R related to the entities at point 2 to the client. This can be accomplished in two ways
    • Add the entities of R as children of the entities of the first entity set they are related to,
    • Move all the entities of R we need as a separate collection and then connect them to the related entities of the first set of entities on the client with the help of the addRelated method of their updateManager (see my previous post).
  4. Define the updateManager of R as child of the updateManager of the first entity set.
  5. Connect the enties of R with the entities of the second entity set with the help of the addRelated method.
  6. To Add a new element to R, add it to the adequate entity of the first entity set, then set the external key of the second entity set, and add to it the pointer to the related entity of the second entity set.
  7. Process the entities of the first entity set + the entities of R with substantially the same techniques we used for the one-to-many relations, by accessing the pointer from each entity in R to the related entity of the second entity set when we need to display the related data.

That’s all for now! In a short time the team of the jsAction project will give us a very easy way to use the updateManager, by inspecting all controllers and providing automatically the right instances of the updateManager already configured to work with each specific controller.

                                               Stay Tuned

                                               Francesco

Tags: , , , , , , ,

Apr 3 2012

Mvc Controls Toolkit Support to Mvc4 WebApi

Category: Asp.net | MVC | WCF | WebApiFrancesco @ 03:57

The new Mvc 4 ApiController, and in particular the possibility it offers to expose an IQueryable on the web simplifies the exchange of data between server and an Html page, a lot.

Now it is possible to pass a dynamically created query from the client to the server with a simple GET operation. We need just to fill the $filter, $orderby, $skip, and $top query string parameters according to the oData protocol.

The upshot.js library may help us on the client side to write the code to interact with the server. Upshot does actually all the job, and make the developer interacts with the data as they were on the client, hiding all hard plumbing, needed to retrieve them, and to update them on the server.

However, the price we pay, to take advantage of all upshot.js features is that we can’t have substantially any business layer on the server side, whose role becomes a kind of “Web Database”.

What if we need a “robust” Business Layer, that manipulates data coming from several collections and/or objects before passing a transaction to the database?

At moment upshot.js contains DataProviders for interacting with:

  1. DbDataController. That is a particular ApiController specialized in interfacing DataBase data.
  2. oData sources. A complete oData source can be implemented with Wcf Data Services. In fact both a generic ApiController exposing an IQueryable, and the specialized DbDataController doesn’t implement the full oData protocol, but just support the $filter, $orderby, $skip, and $top operations, that are the more useful operations in business applications.
  3. Wcf Ria Services. The famous Ria services we use also with Siverlight.

One might think to build a custom DataProvider, and pass it to Upshot.js….However, a similar solution would never give us the “kind of customization” we need in this case. The point is that we can no more look at the server like a kind of “Web DB”, so we cannot use upshot and the DbDataController at all, and we need to proceed manually by building our custom jQuery ajax calls to generic WebApi methods.

However, there is some “hard plumbing job” we can factor out also in this case, namely:

  1. Building oData queries in a more friendly fashion with a LinQ like fluent syntax
  2. Changes tracking on the data. Recording the initial state of our data, so that we can undo all changes performed on them
  3. Computation of the Change Set to send to the server. This way we can save bandwidth by sending to the server just the modifications performed on data, instead of a whole collection of data
  4. Dispatching the errors returned by the sever in the right place in the User Interface, that is near the UI that displays the “wrong” inputs. Possibly, in the right labels rendered by the ValidationMessageFor helpers, and in the ValidationSummary.
  5. Since Json is not able to encode references, that is several properties pointing to the same instance of an object, but just a flat tree of objects, wee need someway to re-compute references links on the client after different collections containing related data have been sent by the server. Some tool that might help us in performing this job on the client might be very useful

Finally if we would like to take full advantage of WebApi also in business applications that manipulate big quantity of data we need to protect someway the IQueryables that we expose on the web to prevent denial of service attacks.

We can decorate our methods with the ResultLimitAttribute to limit the maximum number of items retrieved by the DB, However this is not enough, since a malicious user might try a denial of service attack by requiring a filtering operation on a column without indices of a big DB table. In this case also if no result is found at all the Data Base might waste a lot of time to discover it.

In the new Mvc4 beta compatible version of the Mvc Controls Toolkit we included tools that face all the problems listed above, namely:

  1. The javascript mvcct.Queryable abstract class allow us to build complex queries with a LinQ like fluent syntax. Moreover, it is able to import sorting, filtering, and paging information from all Mvc Controls Toolkit controls. There are three implementation of this abstract class:
    • The mvcct.oData.Queryable to query remote datasources that implements (also partially) the oData protocol. This is the right tool to use to query WebApi that expose an IQueryable
    • The mvcct.upshotQueryable build queries to be executed by the upsjhot.js library.
    • The localQueryable is able to query data that are already on the client in javascript format. It is useful when we need to work offline
  2. The mvcct.updateManager is able to track the changes of a collection of data, to compute the associated Change Set and to post it to a generic WebApi or standard Mvc Action Method. It is also able to transfer the data to the server with a standard browser submit (it creates dynamically a form that it then submits). Moreover it wait for the result from the server, and process possible errors returned by the server, by dispatching them in the right places of the UI. The developer may also provide two optional callbacks to be called respectively in case of success and in case of error. The error processing can be skipped by setting a property contained the callback arguments.
    Several mvcct.updateManager may cooperate to submit several change sets to the server with an unique post. In this case each mvcct.updateManager prepares the Change Set for the collection it takes care of, and put it in the destination ViewModel. Then, just one of them, the master, posts this ViewModel and wait for the server result . Once the result arrives it notifies all other mvcct.updateManager so that each of them can process its errors, and its optional callbacks.
  3. The addRelated method of the mvcct.updateManager computes automatically all mutual references existing between the items of two related collections by using key and external key information.
  4. The SafeQuery and HttpSafequery implementations of the IQueryable interface work as wrappers to protect any IQueryable from unwished queries. The allowed sorting and filering operations are specified through the CanSort Mvc Controls Toolkit attribute. In case a forbidden operation is detected the SafeQuery throws the adequate subclass of the ForbiddenException, while the HttlpSafeQuery that is specialized for the Web throws an HttpResponseException to set the right Http status code.

But let see how all this works in practice with a simple example. The example I use can be downloaded from the Mvc ControlsToolkit download area here: Mvc4 Client- Filtering -Paging -Sorting-updating. The file AdvancedJSonCommunication contains similar examples that works with standard Mvc3 contollers.

As first step let ‘s run the example to see what happens. We can select two pages in the menu: Index, and IndexEdit.The first one is a query and display only example, while the second one gives also the possibility to update the entities. Let start with the Index page.

ToDoDisplay

There is a pager, we can sort by clicking on the columns, and we can filter data by selecting several filtering options simultaneously and then by clicking Refresh

Let select filter by name, and let choose the Contains operator.

ToDoFiltering

Looking for a string contained in any position of a column is a very inefficient type of search if we have not defined a special index called full text index on that column. So it might be used for a denial of service attack against our website. However, when we  hit refresh we get:

ToDoError

A 403 Http status code: FORBIDDEN! The message explains us that the Contains operation is not among the operation allowed on the colum name.

Let see what happened behind the curtain:

  1. public class ToDoController : ApiController
  2.     {
  3.         // GET /api/todo
  4.         public IQueryable<ToDoView> Get()
  5.         {
  6.             return new HttpSafeQuery<ToDoView>(ToDoViewModel.GetToDoQueryable());
  7.         }

We wrapped our IQueryable within the HttpSafeQuery IQueryable that rejected the Contains clause of our filter.

The data annotations on the ToDoView class are provided through a MetaDataType(to understand why I did ths way, give a look here):

  1. [MetadataType(typeof(MetaToDo))]
  2.     public partial class ToDoView
  3.     {
  4.         public int? id { get; set; }
  5.        
  6.         public string Name { get; set; }
  7.        
  8.         public string Description { set; get; }
  9.  
  10.        
  11.         public DateTime DueDate {set;get;}
  12.  
  13.     }

So Let go to the MetaToDo class:

  1. public class MetaToDo
  2.     {
  3.         
  4.         [Required, CanSort, Display(Name="Name")]
  5.         public object Name { get; set; }
  6.         [Required, Display(ShortName = "Description")]
  7.         public object Description { get; set; }
  8.         [CanSort, Display(ShortName = "Due Date"), Format(DataFormatString="{0:D}")]
  9.         public object DueDate { get; set; }
  10.     }

The Name property is decorated with the CanSortAttribute, but since no argument is passed to specify wich filtering options are allowed, the default settings are taken…and the default setting doesn’t allow the dangerous Contains operator.

Now, let give a look to the way the query is built. We created the mvcct.oDataQueryable  object in the View to take advantage of the @Url.RouteUrl method to conpute the Url of our WebApi:

  1. <script type="text/javascript">
  2.     var query = mvcct.oDataQueryable('@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})', mvcct.$$.and);
  3.     query.importSortingControl('@Html.PrefixedId(m => m.ToDoOrder)');//import default sorting (necessary for paging
  4.  
  5. </script>

The second parameter of the constructor is the logical operator to be used to combine all filter conditions. We might have specified also mvcct.$$.or. There is also a third boolean parameter that we have not used in this example, negate. If true the whole filter is negated.

The first operation we do immediately is importing the initial sorting specified in the control that is bound to the ToDoOrder ViewModel property. It was put there by the action method that took care of creating the Index View:

  1. public const int PageDim=5;//in actual application this should be put in a config file
  2.         public ActionResult Index()
  3.         {
  4.             //define default ordering. It is necessary for paging.
  5.             List<KeyValuePair<LambdaExpression, OrderType>> order = new List<KeyValuePair<LambdaExpression,OrderType>>();
  6.             Expression<Func<ToDoView, DateTime>> defaultOrder = m => m.DueDate;
  7.             order.Add(new KeyValuePair<LambdaExpression, OrderType>(defaultOrder, OrderType.Descending));
  8.  
  9.             ToDoViewModel result = new ToDoViewModel()
  10.                 {
  11.                     DataPage = new ToDoPage {
  12.                         ToDoList=new List<ToDoView>()},
  13.                     ToDoOrder=order
  14.  
  15.                 };
  16.             return this.ClientBlockView(result, "ClientToDoView");
  17.         }

As you can see this action method just creates the substantially empty ViewModel that will be used as page ViewModel by the knockout.js library

The adviced way to use knockout.js with the Mvc Controls Toolkit is through the Client Blocks feature. For more information on the sorting Mvc Controls toolkit controls give a look here.

The remaining code used to build the query is in the WebApiRetrieve.js file:

  1. ClientToDoView.Refresh = function (type, page, sorting, prevPage) {
  2.         if (!type) {
  3.             type = 'filter'; page = 1;
  4.             ClientToDoView.DataPage.CurrPage(1);
  5.             //filter changed, we need reset pager to the first page
  6.         }
  7.         query.setPaging(page, pageSize);
  8.         //sorting changed
  9.         //we don't need to reset pager to the first page, sorting control do it automatically
  10.         if (type == 'sort') query.importSorting(sorting);
  11.         else if (type == 'filter') {
  12.             query.resetFilter()
  13.             .importClauses('ToDoFilter');
  14.         }
  15.         query.execute(function (x) {.......

The Refresh method of the ViewModel is called either when the filter, or the sorting or the page is changed, so the first thing it does is “understanding” what changed. Then it imports into the query the changed information. The pager and the sorting controls triggers an event containing the new page and/or sorting information when something changes, so we need just to register an event to handle sorting and paging:

  1. $('#root').bind('queryChanged', function (e, data) {
  2.         ClientToDoView.Refresh(data.type, data.page, data['sortString'], data['previousPage']);
  3.     });

sortString, and previousPage are put within [‘’] because the sortString property is provided just for the sorting changed event, while the previousPage is provided just for the page changed event, so both properties may be undefined.

There is no event that says us the when filtering condition changes, so the best we can do is binding the refresh method of the ViewModel to the Refresh button through a knockout click binding:

  1. <input id="Button1" type="button" data-bind='click: function(x){x.Refresh();}' value="Refresh" />

The mvcct.Queryable.importClauses imports all filtering conditions by inspecting directly all filtering controls that are bound to the  ToDoFilter ViewModel property. For an introduction to the MvcControls Toolkit Filtering controls give a look here.

The execute method issues the query to the WebApi controller, and receives the results through a callback, whose code is not shown.

Basically the callback inspects the format of the object that is returned to see if the total count of all entities is available (this information may be used to improve the paging experience). WebApi methods doesn’t supplies this information and return just a collection of entities, but full implementation of the oData protocol like the Wcf Data services may return it. Then they add observables and observableArray, as needed by calling the ko.mapping.fromJS function of the knockout mapping plugin:

  1. var newEntities = ko.mapping.fromJS(x.results)();

and put them into the ViewModel:

  1. ClientToDoView.DataPage.ToDoList(newEntities);

The total count of the entiies is updated either with the exact information supplied from the server, if available, or with a smart guess:

  1. ClientToDoView.DataPage.TotalPages(totPages);

The pager is bound to the DataPage.TotalPages ViewModel property, and immediately takes advantage of this new information. In case of errors the pager is reset to its previous page:

  1. ClientToDoView.DataPage.CurrPage(prevPage);

The pager is bound to this ViewModel property, and it updates its state immediately. The ko bindings of the pager are automatically created by the pager Html helper.

In case we would like to use the upshot.js library to query our controller, we can stll import paging, filtering, and sorting infos from the Mvc Controls toolkit controls by using the upshotQueryable in place of the the oDataQueryable, since both objects share the same public interface.

The constructor in this case is: mvcct.upshotQueryable(dataSource, fop, options, negate). Where dataSource is an upshot datasource, fop and negate are completely ignored since upshot allows just the logical and of all filter conditions. Finally the option object contains only the includeTotalCount property whose default value is true;

we can also build our query manually with the help of the whole fluent public interface exposed by all mvcct.Queryable objects:

  • setPaging(page, pageSize); Specifies current page and page size. we already used it in our example.
  • addSort(field, desc, enabled): if enabled is true adds a new orderby clause to the order being built. field is the name (actually a string expression) of the property, and the boolean desc specifies if the order is descending.
  • resetSorting():clears all previously added sorting clauses, so that a completely new sorting can be built.
  • addCondition(operator, value1, value2, enabled): if enabled is true a new filtering condition is added to the filter being built. All filter conditions are combined with the currently active logical operator. The initial logical operator is specified in the constructor of the Queryable. A different logical operator can be specified each time we build a complex condition made of several sub-conditions with the help of the open and close methods (see later), value1 is the property to be constrained by the condition,  value2 the second argument of the comparison , and operator is the condition operator  that is one of:
    • mvcct.$$.eq, mvcct.$$.ne, mvcct.$$.gt, mvcct.$$.ge, mvcct.$$.lt, mvcct.$$.le;
    • mvcct.$$.startswith, mvcct.$$.endswith
    • mvcct.$$.substringofInv; that is the Contains operator.
    • mvcct.$$.substringof; that requires the property value is a substring of the value2 argument
  • resetFilter(): clears all previously added filter clauses, so that a completely new filter can be built.
  • open(logicalOperator, enabled, negate): if enabled is true, it starts a complex condition made of several sub-conditions that will be combined with logicalOperator. If negate is true the whole condition will be negated. Its effect is opening a parenthesys in the filter expression being built.
  • close(enabled): close a previously opened parenthesys. enabled must match the enabled of the matching open otherwise the parenthesys of the resulting expression will be unbalanced.
  • importSorting, importSortingControl, and importClauses are the Mvc Controls Toolkit controls importing methods we have already used in our example. There is also an importPager(pagerId, pageSize) method that imports the paging data contained in the pager whose id is pagerId. we have not used it in the example because we took the all page informations from the event triggered by the pager.
  • get(): returns an object whose nature depends on the implementation of the Queryable: the oDataQueryable returns the complete url to be passed to the server, the localQueryable returns a function that once executed performs all operations specified in the query, and the upshotQueryable returns the upshot DataSource.
  • execute(callback): executes the query and passes the result returned by the server as argument to the callback. The upshotQueryable ignores the callback because the only operation performed is the refresh of the DataSource: this is enough to dispatch the server results in the right place.
  • getState() and setState(x) respectively returns an object that encodes the full state of the Queryable, and set the state of the Queryable by using a previously saved state. They are useful to handle the Back and Forward buttons of the browsers to navigate the history of all query passed to the server.

It is time to understand how to handles data updates with the updateManager class. In our example the creation of the  instance of the updateManager that takes care of our ToDo items is in the IndexEdit view:

  1. ClientToDoView.updater = mvcct.updatesManager(
  2.         '@Url.RouteUrl("DefaultApi", new { httproute = "", controller = "ToDo"})',
  3.          ClientToDoView,
  4.          'DataPage.ToDoList',
  5.          'id', null, null, {isoDate:true, updateCallback: function (e, result, status) { alert("status: " + status); } });

The first argument just computes the Url of our WebApi, the second argument is the javascript variable that contains our page ViewModel, the third argument is a string expression that locates the collection we would like to process. The fourth argument is another string expression that locates the “key” of our items. It doesn’t need to correspond to the actual DB key of the item, but it is just the field that we want our updateManager send to the server when an item is deleted.

The further two arguments that we set to null are respectively, the destination ViewModel and a string expression that locates the place in the destination ViewModel where to put the change set once it has been computed. Since both fields are null there is no destination ViewModel, and the change set will be sent to the server, as it is , without inserting it into another container. The destination ViewModel is useful if we need to send simultaneously several change sets to the server.

The change set is an object with just 3 properties: one containing the array of all modified items, another one containing the array of all newly inserted items, and the last property contains the array of the “keys” of all deleted items. The names of these properties can be gonfigured in the option argument of the constructor of the updateManager (the last argument). The default option object is:

  1. {
  2.     updater: {u: "Modified", i: "Inserted", d: "Deleted"},
  3.     isoDate:false,
  4.     updateCallback: function(e, result, status){},
  5.     updatingCallback: function (changes, modelToPost, expr){return changes;}
  6. }

The first property contains the names of the three fields of the change set object.

The second property specifies if the date format to use when sending the change set to the server is the ISO format or the \/Date(….)\/ format. When sending data to a standard Mvc controller we have to set isoDate to false, while in our case we can’t use the default since WebApi controllers need the ISO format.

The updateCallback is called just before processing possible errors returned by the server, and it receives:

  • e. an object with format:
    { setErrors: true, model: sourceViewModel, expression: sourceExpression, key: keyExpression, success: !result.errors }
    If in our callback we set the setErrors property to false, error processing is not carried out, and we have to process them with our custom logics.
  • result. The result returned by the server, that must  be an object containing an error property with the list of all errors and an optional insertedKeys property with the keys created dynamically when inserting new records in the DB. Both the errors and the keys received from the server are processed automatically. That is, the errors are dispatched in the right places of the UI, and the keys to all newly created items that need them.The result object can contain also further properties that can be processed in a custom way  in the updateCallback.
  • status: the Http status of the server response. In our example we specify a callback that just shows the status with an alert window.

The updatingCallback is invoked just before sending all data to the server. If this callback returns a false, null, or undefined value the update is aborted. The updatingCallback receives three arguments:

  • changes: a boolean that informs us if any change was detected
  • modelToPost: the model that the updateManager is going to post
  • expr: a string expression that locates the just computed changes set within the above model.

Let give a look to our receiving action method:

  1. HttpResponseMessage<ApiServerErrors<int>> Post(Updater<ToDoView, int> model)

The Updater generic class that is defined in the MVCControlsToolkit.Controller namespace is just the server side equivalent of our javascript change set class. It is just an helper class since we can use any class to receive the change set. It is enough to specify the right property names in the updateManager option object.

We can use also a complex object containing several change set objects that matchs our javascript destinationViewModel, as parameter  to deal with several simultaneous collection updates. Moreover, we are not limited to using webApi methods but we can use also standard controller action methods.

The full code of the controller method  is:

  1. public HttpResponseMessage<ApiServerErrors<int>> Post(Updater<ToDoView, int> model)
  2. {
  3.     //uncomment to experiment server side error handling
  4.     //ModelState.AddModelError("Modified[0].Name", "Fake error");
  5.     int[] insertedKeys;
  6.     if (ModelState.IsValid)
  7.     {
  8.         insertedKeys=ToDoViewModel.UpdatePage(model.Inserted, model.Modified, model.Deleted);
  9.         
  10.     }
  11.     else
  12.     {
  13.         insertedKeys = new int[0];
  14.     }
  15.     return new ApiServerErrors<int>(ModelState, insertedKeys).Wrap();
  16. }

If the ModelState is valid we call a business layer method, otherwise we abort business computation and return just the list of errors. It is enough to invoke the constructor of the server side equivalent of our javascript result  object passing it the ModelState, and an array with the keys of all newly created items, to create our result object filled with all errors and/or keys. The call to the Wrap method embeds this object into a response containing an appropriate Http status code. In case we have a complex destinationModel with several change sets, we can use a different overload of the constructor that accepts several arrays of keys, each with a string expressoin that specifies the change set of the destinationViewModel it refers to, Namely:

  1. public ApiServerErrors(ModelStateDictionary origin, ApiKeyInfos<T>[] insertedKeys, string prefix = null)

where:

  1. [DataContract]
  2. public class ApiKeyInfos<T>
  3. {
  4.     [DataMember]
  5.     public string destinationExpression { get; set; }
  6.     [DataMember]
  7.     public T[] keys { get; set; }
  8. }

There is also a not generic version of the ApiServerErrors class that we can use when we don’t need to send keys to the client.

The ApiServerErrors class is contained in the MVCControlsToolkit.Controller namespace together with the analogous ServerErrors class that we can use with standard controllers action methods.

Let uncomment the line that creates a fake error and let see how errors are dispatched to the client:

ToDoEditing

Let modify a couple items, and let add a new one, then let hit submitAllChanges.

since the user input was wrong we get a 400 status code (BAD REQUEST):

error400

Then since the property we added the error is: Modified[0].Name, the error is dispatched to the ValidationMessageFor label next to the name of the first modified item:

ValidationError

Since we had no luck with our changes we can decide to undo everything by hitting the Undo All Changes button: Everything is cleared…changes and errors.

How does Undo and Change Set computation work? How to enable them in our applications?

The first step is to prepare all items that we receive from the server:

  1. var newEntities = ko.mapping.fromJS(x.results)();
  2. if (ClientToDoView['updater']) {//if entities may be modified and sent back to the server, prepare them
  3.     ClientToDoView.updater.prepare(newEntities, true);
  4. }

or that we create on the client:

  1. detailToDo.saveAsNew = function () {
  2.     if (!$('#detailForm').validate().form()) return;
  3.     var item = ko.mapping.fromJS({
  4.         DueDate: this.DueDate(),
  5.         Name: this.Name(),
  6.         Description: this.Description(),
  7.         id: null
  8.     });
  9.     ClientToDoView.updater.prepare(item, true);//newly created entity prepare it
  10.     ClientToDoView.updater.inserted(ClientToDoView.DataPage.ToDoList, item);
  11.     this.reset();
  12. };

Then, we must call the inserted method to insert them in our collection as shown above,  the deleted method to delete them:

  1. detailToDo.remove = function (item) {
  2.     this.resetIfSelected(item);
  3.     ClientToDoView.updater.deleted(ClientToDoView.DataPage.ToDoList, item);
  4. };

and the modified method each time the user manipulate them someway:

  1. detailToDo.save = function () {
  2.     var item = this.DetailOf();
  3.     if (!item) return;
  4.     if (!$('#detailForm').validate().form()) return;
  5.     mvcct.utils.restoreEntity(this, item);
  6.     ClientToDoView.updater.modified(item, true, true);
  7.     this.reset();
  8. };

The second and third argument of the modified method say respectively, to prepare the entity if it is not yet prepared(actually we don’t need it since we know entities have been already prepared), and to do an immediate verification of all changes. Thus all properties of our entity are compared with their old values to verify if an actual change occurred, and only if the entity actually changed it is marked as modified. If the third argument of the modified method is false, our entity is marked as modified without performing any immediate verification; the verification is deferred till the time the updateManager compute the change set.

The mvcct.utils.restoreEntity is an utility method of the Mvc Controls Toolkit  that copies an object into another object having its same structure, properly handlig all ko observables. As a default nested objects are visited and their properties copied but arrays are not. However, if its second optional argument is set to true, references to arrays are copied too.

Preparing a property add it two observable properties: _inserted and _modified, that say us the state of the entity. They are used by the routines that compute the change set, but we can use them also to improve the user experience. In our case we bound them to the enabled status of the undo button:

  1. <input type="button" value="Undo" data-bind='click: function(item){detailToDo.undo(item);}, enable: _inserted() || _modified()'/>

If in the call to the prepare method we pass true as second argument(as we have done), changes tracking is turned on, and the entity will remember its initial state. If changes tracking is off no undo is possible, and an entity is marked modified when we call the modified method without performing any check.

Below how to undo a single entity:

  1. detailToDo.undo = function (item) {
  2.     this.resetIfSelected(item);
  3.     ClientToDoView.updater.reset(item);
  4. };

it is enough to call the reset method of the updateManager. The updateManager has also a resetAll method that restore the initial state of the whole collection:

  1. ClientToDoView.undoAll = function () {
  2.     ClientToDoView.updater.resetAll($('#mainForm'));
  3.  
  4. };

It accepts, as argument, a jQuery object containing a form, that it uses to clear all errors. The state of the errors returned by the server can be also cleared manually each time we need by calling the clearErrors(jForm) method of the updateManager. In our example we do it when we perform a new query to the server:

  1. if (ClientToDoView['updater']) {//clear previous errors, since entities shown on the screen will change
  2.     ClientToDoView.updater.clearErrors($('#mainForm'));
  3. }

we can require manually also the dispatching of errors by calling the refreshErrors(jForm, errorState) where errorState is an object with the same format as the result returned by the server. If errorState is null the last errors returned by the server will be used.

The change set can be submitted to the server either by calling the update(jForm) method:

  1. ClientToDoView.save = function () {
  2.     ClientToDoView.updater.update($('#mainForm'));
  3. };

or by calling the submit(jForm) method, in which case a not-ajax normal browser submit is triggered. The form submitted is dynamically created and contains just the destinationViewModel data.Examples showing the use of the submit method and the use of the updateManager with standard controllers action methods are contained in the file Advanced JSon Communication in the download area of the Mvc Controls Toolkit

Both the update and the submit methods have a second argument: isDependent. It is used when submitting simultaneously multiple change sets to the server into an unique destinationViewModem. When isDependent is set to true the updateManager just compute the change set and insert it in the right place in the destinationViewModel without performing any operation on the server. The operation on the server is performed by an unique updateManager, called with isDependent set to false, that acts as Master. When we call the update method the Master dispatches automatically the results returned by the server to the Slave updateManagers, thus causing they update their, entities, their errors, and call their updateCallBack. The array of all Slaves to notify is passed as third argument of the update method.

The addRelated method of the updateManager can help us in handling multiple related collection on entities:

  1. addRelated(collectionExpression, entities, entitiesExternalExpression, inverseCollectionExpression, overrideKeyExpression)

It computes all mutual references existing between the items of two related collections by using key and external key information:

  • entities is the array(or ko observable array) with the collection we would like to relate with the collection handled by the current instance of the updateManager.
  • collectionExpression is a string expression specifying the array contained in each entity of the collection handled by the current instance of the updateManager,  where the pointers to the related entities will be pushed. If the array doesn’t exists it is created, it it already exists it is not cleared.
  • entitiesExternalexpression is a string expression that locates the external key within each entity of the entities collection.
  • inverseCollectionExpression is a string expression specifying the array contained in each entity of the entities collection where the pointers to the related entities will be pushed. If the array doesn’t exists it is created, if it already exists it is not cleared.
  • overrideKeyexpression, if provided, is used in place of the key defined in the current instance of the updateManager.

In a short time I will write a more detailed blog post on how to handle related collections.

All examples on the js Queryable and on the updateManager are contained in the files Mvc4 Client- Filtering -Paging -Sorting-updating and Advanced JSon Communication in the download area of the Mvc Controls Toolkit

In a short time the team of the jsAction project will give us a very easy way to use the updateManager, by inspecting all controllers and providing automatically the right instances of the updateManager already configured to work with each specific controller. For the moment enjoy the post of my Friend Vincenzo of the jsAction team.

Tags: , , , , , , ,