Blog of Joos Buijs

About personal things, process mining and the rest in life.

Posts Tagged ‘mapping

Wanted: Beta testers for XESame

with 2 comments

So, finally, the day is approaching that my baby gets the “1.0” label. But, before I dare to put it out there, I would like to have it tested, and not only by me.

So, what (/who) is XESame?
XESame started as XESma during my Master’s project. The goal of XESame is to extract event logs from data sources. The input format can be database tables, text files or even XML files. The output is an eventlog in the XES or MXML format.
A good slogan for XESame would be: “Opens the cave of process mining wonders”, but that would be a bit bragging.

Err, sounds great but then what?
This event log can be used in ProM to apply process mining analysis (which is sometimes called ‘magic‘ (Dutch article), it also produces very colorful and nice pictures…). More about ProM and process mining can be found at processmining.org.

But why do you want to test it now?
In September 2010, at the BPM’10 conference in New York, the ProM 6 framework will be officially released. Included in this framework is XESame. The next few days and weeks ProM 6 will be tested (internally) for the release. Since XESame will be released for the first time and I’m the only one working on it, I would really like some thorough testing and feedback.

Okay, so, how can I help?
Well, you can do several things, depending on what you like to do and how much time you can/will spend. First of all, I would suggest that you download XESame and try to extract an event log from data you have available. Then report back to me if XESame was useful and why (not).
XESame uses JDBC to connect to the data source. Since I can not test XESame on ‘all’ data source types out there, I’m interested in how it works on different types of data sources (e.g. different databases such as MySQL, Oracle, MS SQL, etc.)
Furthermore, if you encounter any errors, please let me know so I can try to fix them.
I’m also very interested in what features are missing and how XESame can provide better guidance in defining an extraction of event log data.

But I already looked at XESMa, do you need my help?
Well, yes, for two reasons: First, what did you think of XESMa when you tried it? Second: the graphical user interface of XESame is completely different from the (rudimentary and bloated) interface of XESMa. So I always need (and will appreciate) your help.

Okay, so how do I get started?
Good question (and I’m glad that you want to get started).

First of all, you need to download XESame of course and run it. Go to the ProM 6 BPM’10 release page and download the latest version of the framework and XESame. This should be under the section ‘Download’ or otherwise ‘History’.
For Windows users there is an xesame.exe file that you can start. For Mac/Linux/… users start the MainFrame class in the org.processminning.mapper.ui package from xesame.jar.
If you didn’t try XESame or XESMa before, it might be a good idea to read my Master’s thesis (PDF, 8Mb), especially chapters 5 and 6 with all the examples. (Not in any way suggesting that chapters 1, 2, 3, 4 and 7 are not interesting to read of course.) Although the thesis talks about XESMa a lot, everything should also be applicable for XESame.
And if you’re really interested, look at the XESame source code via http://prom.win.tue.nl:8000/Tracsites/browser/public/XESame/src/org/processmining/mapper or point your SVN client to svn://prom.win.tue.nl/public/XESame (you can use “anonymous/anonymous” for anonymous access, although you cannot commit of course).

Once you’re done fiddling around or when you encounter a serious error or bug or get stuck, contact me and I’ll try to help you. The best way to contact me is to go to my employee page and see if you want to come by my office, give me a call or send me an e-mail (or contact me through Office Communicator on my tue mail address).
Unfortunately, I’m only human so on occasion I might be at the restroom, having lunch or even on holiday (from August 9 until (and including) 20).

So, even if you don’t plan to click on any of the above links, I would like to thank you for reading this post. I hope to hear from you soon and until next time,

Update 28-7-2010 16:50 (CET): I forgot to mention that the ‘official home’ of XESame is http://prom.win.tue.nl/research/wiki/xesame/start (I was too exited…).

Joos

P.S. huge disclaimer follows:
Please note that the author, the department or the university can not be held responsible for any damage caused by direct or indirect usage of XESame (or XESMa). It is recommended that XESame will only be provided read access to the data source and that you run XESame on a copy of the data an not on (the only instance of) the original data source. And of course, XESame is not extensively tested (yet) so it might do strange things to you or your computer. But rest assure, me and my computer survived all months of development.

Advertisements

Written by Joos Buijs

July 27, 2010 at 11:30

Master Project Update: the end is approaching

with one comment

Hi all!

Well, the title says it all, the end of my master’s project is in sight!

The application is ‘nearly done’, there are so many things that could be improved but… well, there is not much time. So, let’s say that the application is at the beta stage then. Yesterday I tried to use it on a real data source instead of my toy database of only 10 records. The results were promising and today I’m processing some of the things we encountered yesterday. For instance the parsing of dates into a Java Date instance is problematic. The format of the date is not always the same and automatically detecting the format used is nearly impossible. Therefore the user (/you) can now define the format used to represent the date and time.

Another type of problem we encountered was related to the ODBC driver but that I can not fix… Other improvements are related to me trying to be too smart (which of course turns out wrong). And some performance issues (but these might be related to the ODBC driver used). And of course a lot of small improvements to the user interface can/should be made etc. So much to do, so little time 🙂

Early this week I also ‘finished’ the visualization of the conversion. The idea is to visualize which tables and columns are used in certain attributes. In the screenshot below a very small event log (with one event definition) is visualized. The conversion uses 2 columns of the event.csv table. I know that the visualization shown is very small and larger visualizations will get messy but it’s hard to get it right… And, well, its only a prototype 😉

I’m also working on my thesis, for about a month now. The contents is structured as follows:

  1. Introduction (context, problem, goal, scope and method of the project) [4 pages]
  2. Preliminaries (explanation of process aware information systems (PAIS), event logs, process mining and other conversion tools) [12 pages]
  3. Conversion Aspects (what to consider when defining a conversion) [8 pages]
  4. Solution Approach (how I planned to implement the application) [7 pages]
  5. Solution Implementation (more details of the technical implementation and use of the application) [14 pages]
  6. Case Studies (2 case studies (SAP and a custom system) to show the validity of my application) [to write]
  7. Conclusion (conclusions and future work) [to write]

So, I still have to perform my case studies, write Chapters 6 and 7 plus the abstract, preface etc. and thoroughly read the entire thesis. And all of that within the next 2 to 3 weeks. And then I’ll have to wait for the reviews of my supervisors and prepare for the final presentation of March 29…

You are all invited for my final presentation of course!!! It will be held at March 29 2010 at 15:00 in Eindhoven, the Netherlands. If you like to attend, please let me know then I’ll inform you of the location.

If you can not attend the final presentation and/or want to read my thesis or try out my application, keep an eye on this blog. I’ll post a link to both of them just before or after my final presentation.

So, now I’m going back to programming again (stupid SQL error…) and enjoy the weekend in a little bit.

TTFN!

Joos

Written by Joos Buijs

February 19, 2010 at 17:17

About my master’s project: more concrete

leave a comment »

Since my introduction in my last post about my master project might be a little vague and general I thought that it might be a good idea to provide you with the user interface sketches I made a couple of weeks ago. They are not final of course and some details have changed in the mean time but the main idea stays the same.

Two example screens of my future application at work are:

Mapping application just started with only the log and trace elements.

This screen shows the editor after project creation with only the basic log and trace elements (which can not be added or removed).

Mapping Editor, with events

This screen shows the editor with some events and properties defined.

It might be good to note that you should view the mapping at the ‘meta’ level. This means that we do not define the event instances themselves but we define where to find events for the traces (for which we also defined how to retrieve them). To complicate things further, we might not need to specify each event type (e.g. “Create order”, “pay order”) separately but if we have some kind of event log as input, the event name (or type or WFMElt or what you call it) could be stored in the data source. Then you might only specify one event mapping which retrieves all the event instances from the data source.

The second screen for instance shows the definition of the “Create” event which can be found in the table “Order”. The username and timestamp values can be extracted from the data source as defined in this example. This event would be added to each trace that we can extract according to the trace mapping definition.

Additionally, there are some mapping properties needed for execution. These are the ‘default’ entity to use, how to link two entities together (if not defined in the data source) and a possible selection criteria for traces and/or events. Furthermore, the trace needs to have a unique identifier defined so we can connect events to traces.

The log, trace, event and attribute terms are re-used from the XES definition and the whole mapping definition quite closely follows this event log meta format (where this mapping is another meta level higher I suppose).

Well, I know that this might still sound rather vague but I hope at least less vague then in my previous post.

If you have any questions, please ask!

Joos

PS: I’m actually on holiday this week (this is a scheduled post) so I might not reply before November 16 (2009).

Edit 13-11-2009 21:15: Improved some text, ‘fixed’ the images and added tags to post. Memo to self: never create a post 2 minutes before you leave for a holiday 😉

Written by Joos Buijs

November 10, 2009 at 17:00

About my master project

with one comment

To make sure that this blog won’t be about funny process models alone it might be a good idea to introduce and explain my master’s project subject: Mapping Data Sources to XES in a Generic Way. Let’s dissect this rather vague sentence to explain what it is all about:

Mapping: in this context it means to let the user define a way to map one data source to another.

Data Sources: most people might think of databases first but text files, XML files and even web services can be considered as data sources. Although it must be seen how many data sources we are able to support, we intend to at least support the common database formats and the CSV (comma separated values) plain text format.

XES: the one requiring the most explanation. Although pronounced (as ‘excess’) similar to one of the well known database formats from a well known software vendor, it means something completely different. In this case we refer to the ‘Extensible Event Stream’ format. This format is an extend-able event log format for, well, storing event logs. For people familiar with the MXML format: XES is the new and improved MXML! For people unfamiliar with MXML: visit processmining.org (more specifically, read an informal introduction to MXML (.PPT, 0,9 MB) or read the more formal MXML introduction paper (PDF, 130 KB)). The XES meta-model is implemented in the OpenXES Java library, more information about XES can also be found there.

a Generic Way: of course, we want our application to be applicable in many situations and therefore it must be generic.

So, in brief, the goal of my project is to develop an application that allows a user to define a mapping from a (set of) data source(s) to the XES event log format and to execute this mapping. Resulting in an event log format that can be used for process mining with (the new version of) ProM.

If you have done one or several process mining project you must know that preparing the data is one of the most time consuming (and, in my opinion, most annoying) part of the process mining project. This master project aims at providing an application that will allow you, the process miner, maybe together with the domain expert, to define a mapping from the data source(s) to the XES event log format without the need to write (Java) code.

Any questions, comments, feature requests etc. etc. are more than welcome!!!

See you at the next post! (Which will either be a ‘funny process model’ or a post showing some GUI designs for the application)

Joos

Written by Joos Buijs

November 3, 2009 at 17:00