René Rebe

The wonders of the WinDOwS DOS legacy. Not only does the the command line (cmd.exe) totally suck, one even needs to resort to the new, and not much less sucking powershell for some halfway decent command time scripting:

Measure-Command { cmd /wait /c our.exe args }

Alien, “fun” :-/

For some time I was wondering why WoL (Wake-on-Lan) was not working on a AM2+ Sapphire IPC-AM3DD785G Mini-ITX board with on-board Atheros Attansic (atl1c) that was lingering here for the usual IT testing purposes.

After quite too much time of trial’n error today I finally found out why. Turns out while the BIOS has an APM: “PME Enable” option - that is not enough! In the same power menu it additionally has a rather cryptic: “Control EuP” option that was enabled.

So how should I know what that is? Obviously not a first thought that this somehow relates to wake form standby handling. And it would certainly help if the manual for the board would somewhere prominently and easy to find linked on their product website. Alas, finally with a manual somewhere deeply hidden on their site it just says to all power settings this:

“Power
The Power Management Setup allows you to configure your system to most effectively save energy saving while operating in a manner consistent with your own style of computer use.”

Wonderful details. Anyways - enabled sounds nice, right? Well turns out that option actually disables power to various wake related components for enhanced standby power savings. Look what I found searching for the setting on the Internet in another product’s manual:

“Control EuP - Enables or disables the Energy Using Products (EuP) Ready function. When set to [Enabled], power for WOL, WO_USB, audio and onboard LEDs will be switched off at S5 state”

Ok, great. And with this disabled wake-on-lan finally works on this box, and it can go to the others into the test system rack. Yay!

The Android update lottery is such an annoying thing. I already got a Nexus 5 specifically to get updates at all. But even when there eventually is an over-the-air (OTA) update, one ends up hitting the “check for updates” button for week without actually hitting the update lottery seed win.

So after a week or so of this silly and annoying game I finally side loaded the update over USB, sigh:

./adb devices
./adb reboot bootloader
./fastboot oem unlock
./fastboot flash bootloader …bootloader-hammerhead-hhz12d.img
./fastboot reboot-bootloader
./fastboot flash radio …radio-hammerhead-m8974a-2.0.50.2.21.img
./fastboot reboot-bootloader
./fastboot -w update …hammerhead-lrx21o/image-hammerhead-lrx21o.zip
./fastboot oem lock

What a cluster mess, only silicon valley companies are elite enough to innovate like this, … sigh :-/!

PS: If you get “./adb devices, … # offline” then you probably run a too old adb version, like 1.0.29 instead of the newer 1.0.31 that handles authentication or such, …

PPS: One does not need to install the whole, huge, Gigabytes of Android SDK, there are various sites that offer (ZIP) downloads with just the two CLI Kilobyte tools.

I was just surprised that the “Install OS X Yosemite.app” actually comes with a “createinstallmedia” helper, how nice of them:

$ sudo /Applications/Install\ OS\ X\ Yosemite.app/Contents/Resources/createinstallmedia –volume /Volumes/Install –applicationpath /Applications/Install\ OS\ X\ Yosemite.app –nointeraction
Erasing Disk: 0%… 10%… 20%… 30%…100%…
Copying installer files to disk…
Copy complete.
Making disk bootable…
Copying boot files…
Copy complete.
Done.

For all too long I was wondering why the middle (center, third) mouse button would not work in a Linux VM on VMware Fusion on a Mac. You know, for copy and paste, … in terminals, … all that text ;-)

I actually googled another rainy day, month or winter, but only found more questions than answers (like mouse.vusb.useBasicMouse = “FALSE” and whatsoever).

Today –clicking around in the Mac’s System Preferences– I came arose the setting for the center button that I would normally not use for Mac apps. Turns out setting this to “Button 3″ is just what is needed for VMware to actually get it and pass it down to the Linux VM.

Sometimes solutions can be so simple, sigh!

Update: Hm, only still a problems with the mighty MagicMouse - as it does not allow to configure a “Button 3″ in Apple’s SystemPreferences … Third party hacks apparently floating in the interweb for this :-/

for virtualization and such, on Linux, BSD, Mac OSX (Darwin), Unix:

dd if=/dev/zero of=some-qemu-vm.img bs=64G count=0 seek=1

Update: and resize / growing:

dd if=/dev/zero of=some-qemu-vm.img bs=1G count=0 seek=200

Yesterday I booked a hotel for family coming to visit us in Berlin I got this nice java exception backtrace at the end of the hotel reservation:

HTTP Status 500 -

type Exception report

message

description The server encountered an internal error () that prevented it from fulfilling this request.

exception

javax.servlet.ServletException: BeanUtils.populate
org.apache.struts.util.RequestUtils.populate(RequestUtils.java:467)
org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:798)
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:205)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:415)
javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
de.hrs.web3.web.filter.PasswordChangeFilter.doFilter(PasswordChangeFilter.java:174)
de.hrs.web3.web.trackingswitch.PostViewFilter.doFilter(PostViewFilter.java:69)
de.hrs.web3.web.trackingswitch.sme.PostClickFilterSME.doFilter(PostClickFilterSME.java:67)
de.hrs.web3.web.trackingswitch.PostClickFilter.doFilter(PostClickFilter.java:66)
de.hrs.web3.web.filter.SessionFilter.doFilter(SessionFilter.java:512)
de.hrs.web3.web.filter.CiClientFilter.doFilter(CiClientFilter.java:133)
de.hrs.web3.web.filter.EncodingFilter.doFilter(EncodingFilter.java:80)
de.hrs.web3.web.util.etm.EtmRequestIDFilter.doFilter(EtmRequestIDFilter.java:63)
de.hrs.web3.web.filter.RedirectFilter.performAction(RedirectFilter.java:258)
de.hrs.web3.web.filter.RedirectFilter.doFilter(RedirectFilter.java:236)
de.hrs.web.filter.ExpiresFilter.doFilter(ExpiresFilter.java:225)
de.hrs.web3.web.filter.StaticContentFilter.doFilter(StaticContentFilter.java:105)
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:83)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
de.hrs.web3.web.filter.CiClientSAMLFilter.doFilter(CiClientSAMLFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.metadata.MetadataGeneratorFilter.doFilter(MetadataGeneratorFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLEntryPoint.doFilter(SAMLEntryPoint.java:102)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutFilter.processLogout(SAMLLogoutFilter.java:157)
org.springframework.security.saml.SAMLLogoutFilter.doFilter(SAMLLogoutFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutProcessingFilter.processLogout(SAMLLogoutProcessingFilter.java:169)
org.springframework.security.saml.SAMLLogoutProcessingFilter.doFilter(SAMLLogoutProcessingFilter.java:93)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
root cause

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
java.util.ArrayList.RangeCheck(ArrayList.java:547)
java.util.ArrayList.get(ArrayList.java:322)
org.apache.commons.beanutils.PropertyUtilsBean.getIndexedProperty(PropertyUtilsBean.java:513)
org.apache.commons.beanutils.PropertyUtilsBean.getIndexedProperty(PropertyUtilsBean.java:410)
org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:768)
org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:846)
org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:903)
org.apache.commons.beanutils.BeanUtilsBean.populate(BeanUtilsBean.java:830)
org.apache.commons.beanutils.BeanUtils.populate(BeanUtils.java:433)
org.apache.struts.util.RequestUtils.populate(RequestUtils.java:465)
org.apache.struts.action.RequestProcessor.processPopulate(RequestProcessor.java:798)
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:205)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1164)
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:415)
javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
de.hrs.web3.web.filter.PasswordChangeFilter.doFilter(PasswordChangeFilter.java:174)
de.hrs.web3.web.trackingswitch.PostViewFilter.doFilter(PostViewFilter.java:69)
de.hrs.web3.web.trackingswitch.sme.PostClickFilterSME.doFilter(PostClickFilterSME.java:67)
de.hrs.web3.web.trackingswitch.PostClickFilter.doFilter(PostClickFilter.java:66)
de.hrs.web3.web.filter.SessionFilter.doFilter(SessionFilter.java:512)
de.hrs.web3.web.filter.CiClientFilter.doFilter(CiClientFilter.java:133)
de.hrs.web3.web.filter.EncodingFilter.doFilter(EncodingFilter.java:80)
de.hrs.web3.web.util.etm.EtmRequestIDFilter.doFilter(EtmRequestIDFilter.java:63)
de.hrs.web3.web.filter.RedirectFilter.performAction(RedirectFilter.java:258)
de.hrs.web3.web.filter.RedirectFilter.doFilter(RedirectFilter.java:236)
de.hrs.web.filter.ExpiresFilter.doFilter(ExpiresFilter.java:225)
de.hrs.web3.web.filter.StaticContentFilter.doFilter(StaticContentFilter.java:105)
org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:83)
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
de.hrs.web3.web.filter.CiClientSAMLFilter.doFilter(CiClientSAMLFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.metadata.MetadataGeneratorFilter.doFilter(MetadataGeneratorFilter.java:86)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLEntryPoint.doFilter(SAMLEntryPoint.java:102)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutFilter.processLogout(SAMLLogoutFilter.java:157)
org.springframework.security.saml.SAMLLogoutFilter.doFilter(SAMLLogoutFilter.java:103)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.saml.SAMLLogoutProcessingFilter.processLogout(SAMLLogoutProcessingFilter.java:169)
org.springframework.security.saml.SAMLLogoutProcessingFilter.doFilter(SAMLLogoutProcessingFilter.java:93)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
note The full stack trace of the root cause is available in the Apache Tomcat/7.0.25 logs.

Apache Tomcat/7.0.25 on p-web-www-424.hrs.de/10.250.151.24

First of all nice to reveal their software stack to potential attackers, right? Second, you noticed:

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
java.util.ArrayList.RangeCheck(ArrayList.java:547)
java.util.ArrayList.get(ArrayList.java:322)

“OutOfBoundsException” thank you very much - happy customers to see this matrix gibberish instead of their booking. It are moments like this where I believe a dynamically typed language with “nil” for a non-existing array, and potentially either printing “nil” or just an “” (an empty string) somewhere on some box, or ad, or whatever, nobody probably looks at anyway is a much more elegant way to handle this, …

For App development and debugging I sometimes manual edit our own App’s preference plist and wondered for some time why these tweaks have no effect anymore, …

Turns out since (at least) Mac OS X 10.9 (Mavericks) they are system-wide cached, and synced by some syncdefaultsd (or so), …

Back in the days of PCs and workstations we first had some platform specific UI libs, and then X11, Motif, Gtk+ Qt, Fltk, Fox etc. pp.

Mostly because people were tired of non-portable UI code, having to rewrite most of the App for each and every platform.

And today? Most devs are back to using platform specific code for UIKit, Android, WP8, … insanely sad :-/ Ok, they mostly just instantiate a WebView anyway, … ;-)

We just released another OCRKit milestone release: OCRKit 2.6.

The biggest change in this new version is the switch to our own -written from scratch- PDF library.

Years ago, when we started ExactScan for Mac around 2005 we already wrote PDF ourselves. Back in that day we already had an own image processing library, so why depend on some proprietary, platform specific code? We just implemented writing standard conforming files ourselves. Problem solved.

However, writing PDF files is relatively easy. Reading, and correctly rasterizing PDF files is a really complex challenge. Mostly because the PDF standard is over 1000 pages long, and the various compression formats, encryption, 3D objects, forms, annotations, etc. pp. makes it even more difficult to implement all the various combinations of that.

Initially we were only interested to deliver an awesome OCR, and thus we decided to get the page images by handing the PDF to the Mac OS X frameworks and get the rasterized page back. While this “solved the problem”, it came with some drawbacks. For example we do not know what is on the page. Just one image? Everything black and white? 200dpi, or 300? All implementation details are completely hidden from us. And worst of all the introduction of HiDPI Retina scaling altered the results so that we got 2x scaled and clipped images, and had to rewrite part of the code interfacing with Mac OS X to compensate for this.

In the meantime we are not only interested about Mac apps anymore, more and more customers ask for Windows, and even Linux solutions. And for neither Windows nor Linux we could re-use this PDF interfacing code. For classic Windows there is no system-wide PDF support, and for Linux we could base on some ghostscript, poppler or so.

However, we wanted full control about the feature set, behaviour, and not source in other’s security issues and bugs into our apps, … and something that just works on Windows, too. So we decided we better start our own PDF parsing and rasterization code. It was quite some effort, but the results start to pay off: We have a much, much better understanding of the PDF internals, and more powerful, faster code that just works on any-OS.

Our new Windows version is obviously using it since the beginning (in fact it already processed hundreds of thousands, TBs of PDFs at customers already).

With OCRKit 2.6 the same new code now comes to the Mac version. Improving program behaviour, e.g. not to rely on our previous color detection to determine if a page was black & white, gray or color, and thus retain the exact image appearance, compression, and in some cases vastly speed up the processing. For example on a 2,3 GHz Intel Core i7 15″ Retina MacBook Pro a 44 page test file is decompressed, OCR’ed and re-written in just 16 seconds - down from over 40 seconds in the previous version of OCRKit. This is over two times as fast (and OCRKit already was fast)! And bringing down OCR time on this multi-core machine from approximately one second per page, to 0.36 seconds (yes, zero point three six - nearly a third of a second!) per page!!!

And best of all: OCRKit v2.6 is still a free update for all our existing OCRKit users since 2010. Enjoy and spread the word!

OCRKit - Recognition revisited.