Welcome to my blog, stay tunned :
Home | Blogs | Stephane Eyskens's blog

SharePoint 2013 Preview REST Offering Extension

Hi,

Update 04/2013 : visit this link http://www.silver-it.com/node/130 to download a demo SharePoint-Hosted App that's demoing most of the REST API capabilities
SharePoint 2013 has extended its REST offering in order to provide the external world an easy and standard way to interact with SharePoint objects. In 2010, Microsoft had already made an important step (compared to 2007) by providing both the Client Object Model (CSOM) and a few REST/oData enabled services such as ListData.svc.

With SharePoint 2013, Microsoft pushes way further the REST offering by making it almost as rich as what you can do with the CSOM. BTW, CSOM remains the preferred technique as long as you're dealing with Microsoft Technologies.

However, if you work in a hybrid environment involving both Microsoft and other technologies such as Java for instance, having a standard REST API available can dramatically facilitate the interactions between the two worlds.

In this post, I will shed some light on those new capabilities.

Demo code for download

For the impatients who wouldn't be interested in the explanations below, I built a demo HTML page showing how to:

- Read a site structure and the content of lists & document libraries
- Create and Update Lists
- Create and Update List Items
- Upload documents
- Read the content of a document
- Delete List Items and Lists

Here is a screenshot illustrating this page:

This page is available for download here.

New EndPoint

The new endpoint we can use is _api. This endpoint is an alias for /_vti_bin/Client.svc. Since URL length is limited, Microsoft decided to create the _api alias to shorten the URLs. However, you might see both end points used in future projects so never mind, it's actually the same behind the scenes.

Reading SharePoint 2013 Data

You can address SharePoint objects the following way:

/_api

or this way:

/_vti_bin/Client.svc

Behind the starting point, you can just mimic the CSOM API. That makes it very easy to use for any SharePoint developer. So, say you want to get the metadata of a document whose ID column equals 1, you can write it this way:

/_api/web/lists/getbytitle('Shared%20Documents')/items/getbyid(1)

You notice that you are calling GetByTitle which is a method of the corresponding CSOM ListCollection object. I also used in the above example a call to GetById.

On top of that, the REST API also supports oData, a very popular and generic query engine (I've already posted tons of blog posts on that) which means that you could extend the above query like this:

/_api/web/lists/getbytitle('Shared%20Documents')/items/getbyid(1)?$select=Title,ID

or you could have had an equivalent oData filter (instead of calling the getbyid method :

/_api/web/lists/getbytitle('Shared%20Documents')/items(1)

So, instead of using the more specific SharePoint method names, you'll be able to specify the more generic oData syntax which can be easier for non SharePoint developers. There might be a performance impact however...I've not tested that but it's something to take into account.

So, I guess you've already understood the principle of how this new REST API is working for for those who still didn't, here is an easy way to see how those URLs can be constructed. As I told you, they mimic the CSOM API. So, if you create any type of project (say console), add a reference to Microsoft.SharePoint.Client.dll, you'll easily see the parallelism between the URLs and the CSOM API.

Say for instance, you'd like to get the server relative URL of the rootfolder of list called Shared Documents. With the CSOM, you'd code it this way:

ClientContext ctx = new ClientContext("http://c6208759806/");
Folder TheRootFolder = ctx.Web.Lists.GetByTitle("Shared Documents").RootFolder;
ctx.Load(TheRootFolder, f => f.ServerRelativeUrl);
ctx.ExecuteQuery();
Console.WriteLine(TheRootFolder.ServerRelativeUrl);

if we take more particularly, this line of code :

ctx.Web.Lists.GetByTitle("Shared Documents").RootFolder

we see that we can just convert it to the following URL:

/_api/web/lists/getbytitle('Shared%20Documents')/rootfolder

In the CSOM code, we specified we wanted to retrieve only server relative url, so you can add this to the URL: /serverrelativeurl which will return you the following result:

As you can see, this is pretty easy and powerfull compared to what we could do in 2010.

The same applies to other objects, so you can for instance get the list of folders of a given web just like this:

/_api/web/folders

the list of users like this:

/_api/web/siteusers

Further, I want to know to which group a certain user belongs to:

/_api/web/siteusers/getbyemail('demosp15@silverit.com')/groups

which will return this:

So, I'm not going to elaborate further on reading data but as you can see, we can do much more than before. May be a last thing : you might wonder how to run a caml query? Actually, you'll convert it to an oData filter. So, say that you want to get all documents whose the Title field equals 'test', you'll just write it this way:

/_api/web/lists/getbytitle('Shared%20Documents')/items?$filter=Title eq 'test'

exactly as you would do with ListData.svc which is still there BTW. For just performing CRUD operations on lists & libraries, you might still consider using ListData.svc.

Writing to SharePoint

Getting Digest

For any writing operation, you need to provide SharePoint with a DIGEST token that enforces security between the client and the server. To get the DIGEST value from the system, you first need to perform a query like this:

$(document).ready(function () {   
    $.ajax({
    url: "/_api/contextinfo",
    type: "POST",
    contentType: "application/x-www-url-encoded",
    dataType:"json",
    success: function (data) {
	if(data.d)
	{
	 digest = data.d.GetContextWebInformation.FormDigestValue;			   
	}
 	},
 	error: function (xhr) {
  	 alert(xhr.status + ': ' + xhr.statusText);
 	}
  });
});

On ready, you perform a POST (forbidden with GET) query to contextinfo. This returns JSON object containing all the contextual info you might need. Fiddler outputs this :

{
  "d":{
    "GetContextWebInformation":{
      "__metadata":{
        "type":"SP.ContextWebInformation"
      },"FormDigestTimeoutSeconds":1800,"FormDigestValue":"0xD93B2F7EC3F8E08084F9BB3A7650A0E1F73BA904E78FD44C40D0BDBB49E54DB63A4D3DABD98A15C3837C00CA34145FA821D9DEE4060B883ED2652723B6C88BFC,01 May 2012 08:58:59 -0000","LibraryVersion":"15.0.3612.1010","SiteFullUrl":"http://c6208759806","SupportedSchemaVersions":{
        "results":[
          "14.0.0.0","15.0.0.0"
        ]
      },"WebFullUrl":"http://c6208759806"
    }
  }
}

As you can see from the above returned object, the digest expires after 1800 seconds, meaning that if you try to issue a query after that amount of time, you'll get 403 - Forbidden error.

DELETE operation

Following the same principle than earlier, you can simply run the following AJAX query to delete a list :

$.ajax({
    url: "/_api/web/lists/getbyid('my list')",
    type: "POST",
    contentType: "application/x-www-url-encoded",
    headers: {"accept": "application/json","X-Http-Method": "DELETE","x-requestdigest": digest,"x-requestforceauthentication": true,"If-Match": "*"},      
    success: function () {
        $("#SPLists tr[title='" + list + "']").remove();
          
 	},
 	error: function (xhr) {
  		alert(xhr.status + ': ' + xhr.statusText);
 	}
});

In the above code, I'm using the variable digest that I got earlier. In the above code, I didn't use a DELETE type. While this would have worked, for other operations such as MERGE, you have to use a POST type and complement it with a X-HTTP-METHOD. So, in order to be coherent, I'm using that syntax for every operation.

As you have probably noticed that I'm providing "*" in my if-match header. While this facilitates my job, it's probably not really a best practice. What you're supposed to provide in the if-match header is the ETag value of the resource you are trying to update/delete.

By specifying "*", you will always override the existing resources no matter whether they have already been changed by another process since you acquired them. On the other hand, if you want to work with the right ETag, it means that you must first acquire the resource (list/list item via GET), extracts its Etag and then reuse it when updating/deleting. I'll elaborate further later in this post on how to use ETag properly.

CREATE Operation

Creating a list

As stated in the introduction, the new REST API also allows you to amend the structure of a site. One of these possibilities is by creating a new list.

var jsono={ '__metadata': { 'type': 'SP.List' }, 'BaseTemplate': 100,'Title': list,'Description': '' }
$.ajax({
    url: "/_api/web/lists",
    type: "POST",
      
    headers: {"accept": "application/json","x-requestdigest": digest,"x-requestforceauthentication": true,"If-Match": "*"}, 
    data:JSON.stringify(jsono),     
    success: function (data) {          
        if (data.d) {              
            alert('list created');
        }
 	},
 	error: function (xhr) {
  		alert(xhr.status + ': ' + xhr.statusText);
 	}
});

Creating a list item

To create a list item, it's quite similar to creating a list but there is a small difference though that is not well explained in the SDK.

function AddListItem(TypeName)
{
var jsono = { '__metadata': { 'type': 'SP.Data.' + TypeName + 'ListItem' }, 'Title': title }
    
$.ajax({
    url: "/_api/web/lists/getbyid('"+list+"')/items",
    type: "POST",
    headers: { "accept": "application/json", "x-requestdigest": digest, "x-requestforceauthentication": true, "If-Match": "*" },
    data: JSON.stringify(jsono),       
    success: function (data) {
        if (data.d) {               
             alert(data.d.__metadata.etag);
        }
    },
    error: function (xhr) {
        alert(xhr.status + ': ' + xhr.statusText);
    }
});
}

Indeed, you have to build the type name. In the SDK, they clearly mention that the type name should be:

SP.listnameListItem

but what they didn't specify is that the first letter of "listname" must be uppercase...The easiest way to make sure you always use the right type name is to retrieve the attribute ListItemEntityTypeFullName of each list when retrieving the lists. For doing this, you can use this query in the AJAX query when retrieving the lists:

url: "/_api/web/lists?$select=Title,Id,Description,BaseTemplate,ListItemEntityTypeFullName"

thus, when you'll parse the returned results, you can just pass the value of ListItemEntityTypeFullName as parameter when calling the function AddListItem().

If you don't use a right type name, the HTTP request fails with a 400 status code. As you might have noticed, in the success callback method, I'm reading the value of the returned ETag (see below for more explaination).

Uploading a document

The following function performs the job:

function AddTextFile(list,fname,fcontents)
{
    if (fname == undefined || fname == null || fname == '') {
        alert("Invalid filename");
    }
	$.ajax({	
        url: "/_api/web/lists/getbyid('"+list+"')/rootfolder/files/add(url='"+fname+"', overwrite=true)",
        type: "POST",
        data: fcontents,        
        headers: {"accept": "application/json",	"x-requestdigest": digest,"x-requestforceauthentication": true,"If-Match": "*"},
        success: function () {
            GetListFiles(list);           
        },
        error: function (xhr) {
            alert(xhr.status + ': ' + xhr.statusText);
        }

    });
}

If I call the function this way :

AddTextFile("listguid","afile.txt","dummy file content");

This will create a new document in the target list.

UPDATE Operation

UPDATE with if-match *

The following piece of code updates the description of a list:

var jsono ={ '__metadata': { 'type': 'SP.List' }, 'Description': 'new description'}

$.ajax({
    url: "/_api/web/lists/getbytitle('Documents')",
    type: "POST",      
    headers: { "accept": "application/json", "X-Http-Method": "MERGE", "x-requestdigest": digest, "x-requestforceauthentication": true, "If-Match": "*" },
    data:JSON.stringify(jsono),     
    success: function () {
       alert('update done')
     },
 	error: function (xhr) {
  		alert(xhr.status + ': ' + xhr.statusText);
 	}
});

Here, I'm using the MERGE method. With the if-match set to *, my list will be updated no matter whether it has been updated or not by another process since I acquired it.

UPDATE using a proper ETag

As stated earlier, the ETag is supposed to be used properly in order to handle safe conflicts. A safe conflict occurs when:

  • Process A acquires item A
  • Process B acquires item A
  • Processs A updates item A
  • Process B tries to update item A

When process B will try to update item A, the system should inform him that item A was changed in between. This is what is done with ETag. As said before, if you provide "*" as ETag value, the system will always push your changes whatever happend in between. While this is OK for most scenarios, this might not be for more specific ones. So, I'm going to demonstrate the above list, in code and we'll repeat the above operations to reproduce and handle that problem.

In order to demonstrate this, I'll create two buttons:

<input type="button" value="Get List etag" onclick="GetListWithEtag(prompt('List Title',''))" />
<input type="button" value="Update List with etag" onclick="UpdateListWithEtag(prompt('etag',''),prompt('List Title',''),prompt('New Description',''))" />

The first one will show the current Etag value of a list. We'll use that returned value to click on the second button. Let's now see the implementation of those functions:

function UpdateListWithEtag(etag, list,desc) {

    if (etag == undefined || etag == null) {
        alert('invalid etag');
        return;
    }
    var jsono = { '__metadata': { 'type': 'SP.List' }, 'Description': desc }

    $.ajax({
        url: "/_api/web/lists/getbytitle('" + list + "')",
        type: "POST",
        headers: { "accept": "application/json", "X-Http-Method": "MERGE", "x-requestdigest": digest, "x-requestforceauthentication": true, "If-Match": "\""+etag+"\"" },
        data: JSON.stringify(jsono),
        success: function () {
            alert('List updated!');
        },
        error: function (xhr) {
            alert(xhr.status + ': ' + xhr.statusText);
        }
    });
}
function GetListWithEtag(listname) {
    $.ajax({
        url: "/_api/web/lists/GetByTitle('"+listname+"')",
        type: "GET",
        dataType: "json",
        success: function (data) {
            if (data.d) {
                alert(data.d.__metadata.etag);
            }
        },
        error: function (xhr) {
            alert(xhr.status + ': ' + xhr.statusText);
        }
    });
}

So, now, to reproduce the safe conflict exception, you click on Get List Etag and input the following parameter:

which returns in my case :

Now, you click on the update button and you input the following parameters:



The system will tell you that the list was updated correctly. Now, if you repeat the same operation using the exact same parameters (especially ETag 6), the system will return you the following error:

Because you are using an outdated ETag. Indeed, when you updated the list for the first time, the system had actually returned you the new ETag. So, when you perform an update query, make sure you read the new ETag value, either by getting the response header, either by acquiring a new instance of the updated object. The first solution is better but depends on the browser & jquery version you are using...while second one always works.

To read the returned ETag value after an udpate, you can add this piece of code to your update request:

complete: function (XMLHttpRequest) {
            alert(XMLHttpRequest.getResponseHeader('ETag'));
 }

That works well with Chrome but doesn't with IE9 :). As you probably noticed, unlike when you create a new object, the ETag is not returned in the response body but is returned in the response header. This makes things a bit more complicated to read it.

Other REST APIs

Other REST operations are available in SharePoint 2013, these target Search, User Profiles, Publishing. I'll try to cover them in future blog posts.

Happy Coding!

Comments

Question

Hi, Can you help me with my problem please.
It's related to client.svc but not a developing issue.

I'm using multiple host named site collections and to be able to use people picker in Document Information Panels (like MS Word) or use people picker controls in InfoPath I changed the MultipleSiteBindingsEnabled to True in Web.config file. (As instructed in some articles)

Then people picker controls started working fine in InfoPath and Document Information Panels. However now people and group fields in SharePoint (for example a field in a list or even when adding users to a site) could no pick up a user now and say cannot reach the server.

Checking the event viewer log it says:

WebHost failed to process a request.
Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/37220943
Exception: System.ServiceModel.ServiceActivationException: The service '/_vti_bin/client.svc' cannot be activated due to an exception during compilation. The exception message is: When 'system.serviceModel/serviceHostingEnvironment/multipleSiteBindingsEnabled' is set to true in configuration, the endpoints are required to specify a relative address. If you are specifying a relative listen URI on the endpoint, then the address can be absolute. To fix this problem, specify a relative uri for endpoint 'http://****/_vti_bin/client.svc'.. ---> System.InvalidOperationException: When 'system.serviceModel/serviceHostingEnvironment/multipleSiteBindingsEnabled' is set to true in configuration, the endpoints are required to specify a relative address. If you are specifying a relative listen URI on the endpoint, then the address can be absolute. To fix this problem, specify a relative uri for endpoint 'http://****/_vti_bin/client.svc'.

It seems I should specify relative URI for endpoint client.svc.

How can I create new relative URI?

Thanks in advance

Client.svc

Hi,

I'm not sure that it is supported to modify the web.config of Client.svc! I've encountered similar issues with custom WCF services deployed inside of SharePoint that were built over the built-in WCF factories and I eventually gave up with the idea of having multiple endpoints with SharePoint hosted services;

I have not investigated using the people picker in DIPS...especially now that InfoPath is dead.

Best Regards

file upload

Hi

How would you go and upload an image to the picture library using rest....? I am new to the world of SharePoint, graduate developer...!

Hi, You can have a look at my

Hi,

You can have a look at my demo App showing how to use REST on http://sptoolbasket2013.codeplex.com

Best Regards