René Rebe

Yesterday I booked a hotel for family coming to visit us in Berlin I got this nice java exception backtrace at the end of the hotel reservation:

HTTP Status 500 -

type Exception report

message

description The server encountered an internal error () that prevented it from fulfilling this request.

exception

javax.servlet.ServletException: BeanUtils.populate
org.apache.struts.util.RequestUtils.populate(RequestUtils.java:467)
org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:798)
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:205)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:415)
javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
de.hrs.web3.web.filter.PasswordChangeFilter.doFilter(PasswordChangeFilter.java:174)
de.hrs.web3.web.trackingswitch.PostViewFilter.doFilter(PostViewFilter.java:69)
de.hrs.web3.web.trackingswitch.sme.PostClickFilterSME.doFilter(PostClickFilterSME.java:67)
de.hrs.web3.web.trackingswitch.PostClickFilter.doFilter(PostClickFilter.java:66)
de.hrs.web3.web.filter.SessionFilter.doFilter(SessionFilter.java:512)
de.hrs.web3.web.filter.CiClientFilter.doFilter(CiClientFilter.java:133)
de.hrs.web3.web.filter.EncodingFilter.doFilter(EncodingFilter.java:80)
de.hrs.web3.web.util.etm.EtmRequestIDFilter.doFilter(EtmRequestIDFilter.java:63)
de.hrs.web3.web.filter.RedirectFilter.performAction(RedirectFilter.java:258)
de.hrs.web3.web.filter.RedirectFilter.doFilter(RedirectFilter.java:236)
de.hrs.web.filter.ExpiresFilter.doFilter(ExpiresFilter.java:225)
de.hrs.web3.web.filter.StaticContentFilter.doFilter(StaticContentFilter.java:105)
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:83)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
de.hrs.web3.web.filter.CiClientSAMLFilter.doFilter(CiClientSAMLFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.metadata.MetadataGeneratorFilter.doFilter(MetadataGeneratorFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLEntryPoint.doFilter(SAMLEntryPoint.java:102)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutFilter.processLogout(SAMLLogoutFilter.java:157)
org.springframework.security.saml.SAMLLogoutFilter.doFilter(SAMLLogoutFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutProcessingFilter.processLogout(SAMLLogoutProcessingFilter.java:169)
org.springframework.security.saml.SAMLLogoutProcessingFilter.doFilter(SAMLLogoutProcessingFilter.java:93)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
root cause

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
java.util.ArrayList.RangeCheck(ArrayList.java:547)
java.util.ArrayList.get(ArrayList.java:322)
org.apache.commons.beanutils.PropertyUtilsBean.getIndexedProperty(PropertyUtilsBean.java:513)
org.apache.commons.beanutils.PropertyUtilsBean.getIndexedProperty(PropertyUtilsBean.java:410)
org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:768)
org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:846)
org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:903)
org.apache.commons.beanutils.BeanUtilsBean.populate(BeanUtilsBean.java:830)
org.apache.commons.beanutils.BeanUtils.populate(BeanUtils.java:433)
org.apache.struts.util.RequestUtils.populate(RequestUtils.java:465)
org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:798)
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:205)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:415)
javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
de.hrs.web3.web.filter.PasswordChangeFilter.doFilter(PasswordChangeFilter.java:174)
de.hrs.web3.web.trackingswitch.PostViewFilter.doFilter(PostViewFilter.java:69)
de.hrs.web3.web.trackingswitch.sme.PostClickFilterSME.doFilter(PostClickFilterSME.java:67)
de.hrs.web3.web.trackingswitch.PostClickFilter.doFilter(PostClickFilter.java:66)
de.hrs.web3.web.filter.SessionFilter.doFilter(SessionFilter.java:512)
de.hrs.web3.web.filter.CiClientFilter.doFilter(CiClientFilter.java:133)
de.hrs.web3.web.filter.EncodingFilter.doFilter(EncodingFilter.java:80)
de.hrs.web3.web.util.etm.EtmRequestIDFilter.doFilter(EtmRequestIDFilter.java:63)
de.hrs.web3.web.filter.RedirectFilter.performAction(RedirectFilter.java:258)
de.hrs.web3.web.filter.RedirectFilter.doFilter(RedirectFilter.java:236)
de.hrs.web.filter.ExpiresFilter.doFilter(ExpiresFilter.java:225)
de.hrs.web3.web.filter.StaticContentFilter.doFilter(StaticContentFilter.java:105)
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:83)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
de.hrs.web3.web.filter.CiClientSAMLFilter.doFilter(CiClientSAMLFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.metadata.MetadataGeneratorFilter.doFilter(MetadataGeneratorFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLEntryPoint.doFilter(SAMLEntryPoint.java:102)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutFilter.processLogout(SAMLLogoutFilter.java:157)
org.springframework.security.saml.SAMLLogoutFilter.doFilter(SAMLLogoutFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutProcessingFilter.processLogout(SAMLLogoutProcessingFilter.java:169)
org.springframework.security.saml.SAMLLogoutProcessingFilter.doFilter(SAMLLogoutProcessingFilter.java:93)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
note The full stack trace of the root cause is available in the Apache Tomcat/7.0.25 logs.

Apache Tomcat/7.0.25 on p-web-www-424.hrs.de/10.250.151.24

First of all nice to reveal their software stack to potential attackers, right? Second, you noticed:

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
java.util.ArrayList.RangeCheck(ArrayList.java:547)
java.util.ArrayList.get(ArrayList.java:322)

“OutOfBoundsException” thank you very much - happy customers to see this matrix gibberish instead of their booking. It are moments like this where I believe a dynamically typed language with “nil” for a non-existing array, and potentially either printing “nil” or just an “” (an empty string) somewhere on some box, or ad, or whatever, nobody probably looks at anyway is a much more elegant way to handle this, …

Why, oh my, is it necessary to run a 10 MB, 1:11m, 720p (no less) video on the main PayPal website front page background. You know, the page where people go to login, check their balance, and such, …

You wonder the internet is slow? Getting slower and slower? Your monthly “flatrate” (*lol*) volume used up faster and faster?

Wonder no more:

So for about a decade the NSA and it’s five or so dwarf friends focussed on reading all our email, half naked holiday beach posts on Facebook, business contracts and CAD drawings send by email, every appointment for dinner with your girlfriend or wive (or both) and such, …

… and they can not even predict the annexation of Crimea or a war in Ukraine coming, nor find the missing MH370 plane, or show actual evidence who shot the MH17 plane, nor a sign of AH5017.

Now that are priorities, right? All our email, and most personal (and thus usually unimportant to others) communiques - however, the actual real threads, like, you know, war, soldiers, weapons, rockets, and lost planes?

Brave new world - probably no espionage to gain from that, … :-/ !!!

So the skills of the NSA in this day and age boil down to: ☑ reading emails, ☑ watching nude citizens, … But unfortunately not the classic spy skill set like, ☐ robbing thru the dessert, ☐ monitoring actual criminals, ☐ and such … I suggest they watch one or another James Bond movie as free training material, …

So now NEC finally comes out with a 4k IPS display. Unfortunately they lost me as a customer already early this year, …

Wondering if they also use MST, and how the compatibility is with Mac OS X ;-)

For App development and debugging I sometimes manual edit our own App’s preference plist and wondered for some time why these tweaks have no effect anymore, …

Turns out since (at least) Mac OS X 10.9 (Mavericks) they are system-wide cached, and synced by some syncdefaultsd (or so), …

Back in the days of PCs and workstations we first had some platform specific UI libs, and then X11, Motif, Gtk+ Qt, Fltk, Fox etc. pp.

Mostly because people were tired of non-portable UI code, having to rewrite most of the App for each and every platform.

And today? Most devs are back to using platform specific code for UIKit, Android, WP8, … insanely sad :-/ Ok, they mostly just instantiate a WebView anyway, … ;-)

We just released another OCRKit milestone release: OCRKit 2.6.

The biggest change in this new version is the switch to our own -written from scratch- PDF library.

Years ago, when we started ExactScan for Mac around 2005 we already wrote PDF ourselves. Back in that day we already had an own image processing library, so why depend on some proprietary, platform specific code? We just implemented writing standard conforming files ourselves. Problem solved.

However, writing PDF files is relatively easy. Reading, and correctly rasterizing PDF files is a really complex challenge. Mostly because the PDF standard is over 1000 pages long, and the various compression formats, encryption, 3D objects, forms, annotations, etc. pp. makes it even more difficult to implement all the various combinations of that.

Initially we were only interested to deliver an awesome OCR, and thus we decided to get the page images by handing the PDF to the Mac OS X frameworks and get the rasterized page back. While this “solved the problem”, it came with some drawbacks. For example we do not know what is on the page. Just one image? Everything black and white? 200dpi, or 300? All implementation details are completely hidden from us. And worst of all the introduction of HiDPI Retina scaling altered the results so that we got 2x scaled and clipped images, and had to rewrite part of the code interfacing with Mac OS X to compensate for this.

In the meantime we are not only interested about Mac apps anymore, more and more customers ask for Windows, and even Linux solutions. And for neither Windows nor Linux we could re-use this PDF interfacing code. For classic Windows there is no system-wide PDF support, and for Linux we could base on some ghostscript, poppler or so.

However, we wanted full control about the feature set, behaviour, and not source in other’s security issues and bugs into our apps, … and something that just works on Windows, too. So we decided we better start our own PDF parsing and rasterization code. It was quite some effort, but the results start to pay off: We have a much, much better understanding of the PDF internals, and more powerful, faster code that just works on any-OS.

Our new Windows version is obviously using it since the beginning (in fact it already processed hundreds of thousands, TBs of PDFs at customers already).

With OCRKit 2.6 the same new code now comes to the Mac version. Improving program behaviour, e.g. not to rely on our previous color detection to determine if a page was black & white, gray or color, and thus retain the exact image appearance, compression, and in some cases vastly speed up the processing. For example on a 2,3 GHz Intel Core i7 15″ Retina MacBook Pro a 44 page test file is decompressed, OCR’ed and re-written in just 16 seconds - down from over 40 seconds in the previous version of OCRKit. This is over two times as fast (and OCRKit already was fast)! And bringing down OCR time on this multi-core machine from approximately one second per page, to 0.36 seconds (yes, zero point three six - nearly a third of a second!) per page!!!

And best of all: OCRKit v2.6 is still a free update for all our existing OCRKit users since 2010. Enjoy and spread the word!

OCRKit - Recognition revisited.

So we learned the NSA, and their FVEY (Five Eyes) friends plus other fellows such as the German BND et al. are effectively making their own copy of all the data going thru their optical fibre splitters and such.

Now all this is of course seriously bad, unconstitutional, and exactly creating the police state Orwell already imaging in the now famous 1984 and we could already see in the East Germany’s (GDR) Stasi.

This anti-democratic setup and questions aside, … they effectively duplicating most (if not all, minus the youtube video streams) data on the internet.

Now, imagine, just for a brief unlikely moment, they would stop doing this. This would effectually freeing up a whole lot of bandwidth, like double the Internet capacity. Make everything fast and snappy. Imagine how many 4k video streams that would be!!!

And, actually, I also wonder how many connection issues all this surveillance cause. Certainly not the optical fibre splitters, but other kind of non-optical duplication law inspection certainly causes some connection drops left and right. And yes, I have seen proprietary commercial firewall code, … ! :-/

Yes, there have been long enough flamewars to no end, and we know where Minix and Linux stand today in regards to the installed device base, … however, modern Windows NT, and Mac OS X are a bit micro kernel’ish to some degree, …

In wake of last twelve months (and counting) NSA, GCHQ & Co revelations let’s look at the processes running on a typical network appliance:

PID Uid VmSize Stat Command
1 root 364 S init
2 root SW [keventd]
3 root RWN [ksoftirqd_CPU0]
4 root SW [kswapd]
5 root SW [bdflush]
6 root SW [kupdated]
8 root SW [mtdblockd]
35 root SWN [jffs2_gcd_mtd2]
87 root 364 S logger -s -p 6 -t
89 root 364 S init
96 root 376 S syslogd -C 16
99 root 348 S klogd
230 root 388 S udhcpc -b -p /var/run/udhcpc.eth0.1.pid -i eth0.1
314 root 388 S /usr/sbin/dropbear -g

Hm, ok, so aside the minimal logging, dhcp server and the dropbear SSH server for administration tasks we got nothing separated in the user-mode context. All the networking, packet filtering, firewalling, load balancing, WiFi stack and what not is all running in the kernel context.

Yeah, right, exactly that kernel context where a typo, off-by-one, etc. pp. likely sooner than later crashes (oops) the whole system, or gives you a root login.

Would it not be nice if such a typo, bug, … in the NIC driver, the IPv4 or v6 stack, or firewall, or mostly anywhere else would just segfault, and restart an associated user-space ipv4d, iptabled, hosted?

With more isolated drivers and sub-system we certainly should have rather less security issues, and given Linus’ famous performance quotes - I do rather trade some percent of performance for more security. Besides, nowadays we run most systems virtualized with even more performance overhead, … for security, management and scalability.

I kinda like some of the innovations of the Surface 3, like the kickstand, improved keyboard. What I like about the Surface 3 is that the keyboard is cool as the tablet with it’s CPU/GPU behind the display gets warm there. Leaving my typing experience without heated fingers, … Of course the stylus input is nice to have.

One major drawback, however, is that due to the gab between the tablet and the kickstand one can not really use the Surface’s on airline seat desks, … :-/

However, what is a complete shame at this date and time is the fact that the Surface’s are absolutely not repairable.

Of course I do not expect to upgrade the CPU, nor RAM in such laptops. But upgrading the storage (SSD) in a year or two (or when all it’s write cycles are exhausted) or to swap in a new battery in a similar timeframe, … (each of my classic MacBooks got a second battery by now, … in some even the second battery died since then, …). This is not asked too much, and even a must considering todays level of environmental pollution and landfills, …

A tablet / laptop like this can be for good use, even second hand, for five if not ten years. A glued construction like this seriously limits this possibilities without any good reason. Beside maximum company profits, … of course.