Data retrieval with datajs and Server paging size problem

Jul 5, 2011 at 12:28 PM

Hi,

I have a question regarding datajs and retrieving data from northwind db:

When i use datajs to retrieve:

http://services.odata.org/Northwind/Northwind.svc/Products?$skip=0&$top=100&$inlinecount=allpages

I only get the first 20 entries and then at the bottom of the server XML response:

  <link rel="next" href="http://services.odata.org/Northwind/Northwind.svc/Products?$inlinecount=allpages&amp;$top=80&amp;$skiptoken=20" />

I guess the reason it that the server paging is set to 20 entries.
How can I retrieve the desired 100 entries at once with datajs without doing a followup read request? Or how can I get the info that not all data was loaded?

Do I have to use caching with datajs or does it also work without caching and the normal OData.read function?

 Thank you!

Jul 5, 2011 at 6:42 PM

Joe, you're right in that the next link indicates that there are additional results.

The server ultimately controls the amount of information returned, so you really just have a couple of options.

1. Set the "max data service version" header to 1.0. The server may realize that you won't understand the "next" link and decide not to paginate.

2. A better choice is to simply keep reading until you have enough records for whatever purpose you intended.

For a sample of #2, see the "Read paginated data" snippet at http://datajs.codeplex.com/wikipage?title=OData%20Code%20Snippets. If you don't want to keep reading until the end, you can keep a counter around and "stop early" once you've got enough data.

Hope this helps,

Marcelo

Jul 7, 2011 at 6:52 AM

Thanks Marcelo,

 

I haven't tested it yet but would it work when using the cache with a read range of 100 like this?:

cache.readRange(0, 100).then(function (data) {
$.tmpl($("#my-template"), data).appendTo("#somewhere");
});

Thanks

Jul 7, 2011 at 9:32 PM

Hi Joe,

When using the cache you don't have to worry about server side paging. The cache implementation handles this for you.

Regards,

Alex Trigo.

Jul 8, 2011 at 7:09 AM

Hi Alex,

that's what I wanted to hear :)

Thanks!

Jul 8, 2011 at 9:08 AM

Another question...

 

what's when using a proxy servlet ... so my client request url would be something like this:

http://server/proxyservlet/services.odata.org/Northwind/Northwind.svc/Products?$skip=0&$top=100&$inlinecount=allpages

 In this case...the additional loading won't work because the URL

<link rel="next" href="http://services.odata.org/Northwind/Northwind.svc/Products?$skiptoken=20" />

would not go through the proxy...

What could I do then? The same problem occurs when using datajs caching

Will datajs support proxies?

I saw in the code (queryForDataInternal function) that you also take the __next URLs

Or is there another solution?

 

Jul 11, 2011 at 6:43 PM

Hi Joe,

For the proxy servlet scenario your best bet would be to override the default httpClient and implement your own, which forwards requests to the proxy. Then the rest of your app can work with the unproxied URLs, the server will also return unproxied URLs in its payloads, and the httpClient will take care of the proxying.

Take a look at this page for more information on how to override the httpClient: http://datajs.codeplex.com/wikipage?title=OData%20Networking

Please let us know if you are able to get that to work!

David

Jul 14, 2011 at 2:35 PM

Hi David,

great, I got it to work with the help of your Custom OData httpClient sample.

You guys rock :D

Best regards

Joe

 

 

 

Jul 19, 2011 at 12:07 AM

Glad to hear it!