WAQL-PP: Preprocessor for a Data Aggregation Query Language

This week I started to design and implement a preprocessor for the Web-service Aggregation Query Language (WAQL) which is an extension of XQuery. This language is used as part of the WS-Aggregation framework developed at the Distributed Systems Group of the Vienna University of Technology. With this text I want to explain the motivation behind WAQL and how the preprocessor will be designed. The motivation is nicely stated as part of my task description.

The key idea of WAQL is that it provides a convenience syntax for XQuery, which otherwise tends to become complex and hardly comprehensible in bigger scenarios. WAQL queries are transformed into valid XQuery expressions, which are finally executed by a (third-party) XQuery engine.

First of all we need to get a grasp of what the WAQL extensions to the XQuery language are. Since WAQL is still in its experimental stages, there is no exact specification of the language and it may change or grow over time. At the moment WAQL consists of two language constructs:

  • Template Lists: This extension tries to simplify the specification of generated inputs. It basically is syntactical sugar representing a XQuery for-loop construct and as such can be transformed easily.
  • Data Dependencies: This second extension is the interesting one, it can express dependencies between several different queries. The framework has to identify these dependencies and execute the queries in a valid order, so that all dependencies can be resolved.

The above two constructs should explain why the actual transformation has to be split into several phases which can be triggered by the framework at different points in time. The separate steps the preprocessor has to perform are as follows:

  1. Parsing: The textual WAQL query is parsed and an intermediate representation is constructed. Since WAQL is an extension which enhances the set of expressions for the XQuery language, the actual parser has to understand the full XQuery grammar. This may sound like a lot of work, but the XQuery specification provides a detailed description of the grammar in about 140 EBNF rules. So defining a valid parser is a doable job.
  2. Resolving of data dependencies: At this point the preprocessor has generated a list of all unresolved data dependencies. However the preprocessor has no idea which other queries are linked to the one currently being processed. So the actual resolving has to be done by the framework, the preprocessor just adapts the intermediate representation to the data provided by the framework.
  3. Transformation: Once all dependencies have been resolved the intermediate representation can be transformed back into a textual XQuery (without any WAQL extensions), which can then be passed on to a third-party XQuery engine.

Now that the basic operations are defined, we are able to give a rough description of the WAQL preprocessor and how it can be embedded into the existing framework. The two basic modules are a generated parser (obviously performing the parsing step) and a driving engine (performing the resolving and transformation steps). The parser will most certainly be generated using the JavaCC parser generator. The below graphic should explain the architecture.

Architecture of the WAQL preprocessor

Note that the above explanation is written from the compiler-constructor point of view, it just covers the preprocessor as part of the framework. All the other nasty details of WS-Aggregation are beyond the scope of this text. If you are interested you should read the paper or contact Waldemar Hummer who was kind enough to explain it to me. Also I will continue to write about the ongoing development of the preprocessor, so stay tuned.

Update: This text was crossposted to the DSG Praktika Blog as well.

It is a great WAQL-PP

It is a great WAQL-PP Preprocessor for data and I like the processes it made! It's so smooth and I am interested to have more updates. Thanks!

I really enjoyed this site.

I really enjoyed this site. This is such a Great resource that you are providing and you give it away for free. It gives in depth information. Thanks for this valuable information.

This is such a great blog.

This is such a great blog. Thank you for sharing your talent with everyone. You are an inspiration.

You guys are doing a great

You guys are doing a great job. Thank you, and keep it up!

Thanks for your thoughts, I

Thanks for your thoughts, I agree with you.

I agree with your thoughts!

I agree with your thoughts!

What's up to all, the

What's up to all, the contents present at this website
are actually awesome for people knowledge, well, keep up the good work fellows. https://www.checkli.com/john000098

I appreciated your effort in

I appreciated your effort in doing this. You are helping to educate people who have an interest in this field. I salute you for this man!

Ditto! This was such a well

Ditto! This was such a well written article, I hope more keeps coming.

I was impressed with how you

I was impressed with how you explained and presented it in an understandable manner for a newbie like me.

I completely agree. I am so

I completely agree. I am so new to this so it helped a ton!

I'm taking notes every time

I'm taking notes every time you have this kind of post. We are in the same field but I am still learning.

I am the same way! I've got a

I am the same way! I've got a notepad open at all times to make sure i never miss anything!

Pingback

[...] of WAQL-PP, a WAQL Preprocessor for Java I was working on for the last two weeks. In one of the former posts I described the motivation behind this little project and how I planned to implement it. I’m [...]

I think that everything wrote

I think that everything wrote was actually
very logical. However, think about this, suppose you wrote a catchier
post title? I am not suggesting your content is not good., but what if you added a post title to possibly get folk's attention? I mean Reply to comment
| antforge.org is kinda plain. You might glance at Yahoo's front page and note how they create article titles
to get people to click. You might add a related video or a picture or two to grab readers excited about what you've written. In my opinion, it would make your posts a little bit more interesting. https://www.40billion.com/profile/733054835

Pingback

[...] text was crossposted to my personal blog as well. ShareSave Tagged as: JavaCC, WAQL, WS-Aggregation, XQuery Leave a [...]

Everything is very open with

Everything is very open with a precise description of the challenges.
It was really informative. Your website is useful. Thanks for sharing! https://www.coursera.org/user/b13e2339a95bd40c3d4575a70d1dfafe

Thank you for sharing your

Thank you for sharing your site as well keto! You did amazing.

Taking a look at your

Taking a look at your services, it is awesome. From time to time, we do need services to repair our appliances.
and if you ever need photography of Newborns in Portland OR, hit us up @ https://www.portlandbabyphotographer.com/.

Pingback

[...] of WAQL-PP, a WAQL Preprocessor for Java I was working on for the last two weeks. In one of the former posts I described the motivation behind this little project and how I planned to implement it. I’m [...]

Pingback

[...] post:  WAQL-PP: Preprocessor for a Data Aggregation Query Language … By admin | category: VIENNA University of Technology | tags: distributed, language, [...]

Now are you talking about the

Now are you talking about the WAQL-PP.2.0 processor or the Vienna University technology professors?

whoah this blog is wonderful

whoah this blog is wonderful i love reading your posts.
Keep up the good work! You already know, many individuals
are looking round for this info, you can help them greatly. https://angel.co/u/james-land-1

Yeah, right? I love reading

Yeah, right? I love reading this too! so informational! IF you are looking for angels, take a look at our gallery of newborns! They are literally so adorable like an angel.
check us out at www.sanjosebabyphotos.com/