OTN EMEA tour 2015

I am thrilled to be able to go on another OTN tour this year. As a successor to the 2014 MENA tour to Tunisia, Saudi Arabia and Dubai we are organizing a true EMEA tour that visits all three continents of the region. Together with my fellow ACE Directors Heli, Debra and Tariq we will visit Lisbon, Amman and Johannesburg in a week filled with presentations, networking and a lot of travel. You can read more about the tour at otnemea.com.

Unfortunately, the tour will “only” be three stops this year but we are already talking to more local sponsors about adding more new locations to the next tour. And if you are interested in hosting an event as part of an EMEA OTN tour in your country, please let me know.

find database upgrade dates in AWR

Just a really tiny quick tip. Today I was wondering when exactly we performed upgrades on a particular database. Luckily, there is a view in AWR for that.

SQL> select version, min(startup_time) from dba_hist_database_instance group by version order by 1 desc;

----------------- --------------------------        09-APR-14 AM        16-JAN-13 PM        09-MAY-11 PM        22-JAN-10 PM

debugging puppet and facter with strace

I worked on a slightly annoying problem yesterday and thought I’d share the process with you. On one of our linux boxes puppet stopped working with an error that was not very helpful:

[root@portrix]# puppet agent --test
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve local facts: private method `split' called for nil:NilClass
Error: Failed to apply catalog: Could not retrieve local facts: private method `split' called for nil:NilClass

All this told me was that something was wrong with facter during the puppet run. I was able to reproduce the error with just facter but even the debug output was not much more helpful:

[root@portrix]# facter --puppet --debug
Found no suitable resolves of 1 for ec2_metadata
value for ec2_metadata is still nil
private method `split' called for nil:NilClass

In fact, this was even more misleading since I was drawn to the message about ec2_metadata. This box is a physical server and should not have anything to do with ec2. I chased this rabbit for a while but without any success. The best idea then became to run strace on the command and see if this returned anything useful. Strace records and prints all system calls that are made during the execution of a program and among other things it will show you which libraries, files etc are being touched. I produces a ton of output and I will spare you the whole thing. But the general idea is to scan the output from the bottom up. The first things (from the bottom) will be the printing of the error message to your screen and hopefully the stuff that caused the error will show up slightly above that. This is the (not very interesting) part where the error message is printed:

[root@portrix]# strace facter --puppet --debug
execve("/usr/bin/facter", ["facter", "--puppet", "--debug"], [/* 28 vars */]) = 0
brk(0)                                  = 0x18d9000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f78f4f7a000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
[... many, many lines deleted ...]
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
write(2, "private method `split' called fo"..., 46private method `split' called for nil:NilClass) = 46
write(2, "\n", 1
)                       = 1
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigaction(SIGINT, {SIG_DFL, [INT], SA_RESTORER|SA_RESTART, 0x30162326a0}, {0x3c6a295ef0, [INT], SA_RESTORER|SA_RESTART, 0x30162326a0}, 8) = 0
close(3)                                = 0
munmap(0x7fe785c7e000, 4096)            = 0
exit_group(12)                          = ?

And just a bit above that is the last file it read before failing:

stat("/etc/oratab", {st_mode=S_IFREG|0664, st_size=741, ...}) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
open("/etc/oratab", O_RDONLY)           = 4
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
fstat(4, {st_mode=S_IFREG|0664, st_size=741, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe785c7c000
read(4, "#\n\n\n\n# This file is used by ORAC"..., 4096) = 741

So it read my the /etc/oratab file and if we scan the output just a bit further up we can also find the module that does this:

stat("/var/lib/puppet/lib/facter/oracle_sid.rb", {st_mode=S_IFREG|0644, st_size=255, ...}) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
open("/var/lib/puppet/lib/facter/oracle_sid.rb", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=255, ...}) = 0
close(4)                                = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
open("/var/lib/puppet/lib/facter/oracle_sid.rb", O_RDONLY) = 4
close(4)                                = 0
open("/var/lib/puppet/lib/facter/oracle_sid.rb", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=255, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe785c7c000
read(4, "require 'facter'\n\nisOracle = Fil"..., 4096) = 255
read(4, "", 4096)                       = 0
close(4)                                = 0

This file oracle_sid.rb is a facter plugin that we wrote ourselves and immediately when I saw this I knew where to look. This is the part of our code that caused this error in facter:

result = open('/etc/oratab') { |f| f.grep(/dbhome_1/) }
result = result[0].split(":").first

The code is looking for the first line with the string “/dbhome_1″ in it (which usually works for our databases) and then extracts the oracle SID from that line (seperated by colons). But the oratab on that server was empty, so we were trying to split an empty string, resulting in the error “private method `split’ called for nil:NilClass”.

Once this was known the fix was rather easy and this is not what I wanted to show in the blog post. But I hope you got the idea of using strace to debug a linux program that is not returning the level of logging that you would need to find the error yourself.

UKOUG Tech14 Slides: Testing Jumbo Frames for RAC

Just a quick post to link to the slides from my presentation at the UKOUG Tech14 conference. The slides do not appear to be visible on the Super Sunday site, hence this post. The presentation was called “Testing Jumbo Frames for RAC”, the emphasis as much on the testing as on Jumbo Frames.


A discussion on the usage of Jumbo Frames for the RAC interconnect. We’ll cover background information and how this can be tested at the Oracle level and in the Operating system. We’ll also discuss how testing initially failed and why it’s interesting.

This is a topic that enters the realm of network and Unix admin’s but the presentation is aimed at DBAs who want to know more and want to know how to use Operating System tools to investigate further.




The conference itself was another success with particular highlights for me being James Morle talking about SAN replication vs DataGuard, Ludovico Caldara talking about PDBs with MAA, Richard Foote because I was like a silly little fanboi and Bryn Llewellyn purely for the way he weaves the English language in to something beautiful on the ears. All of my programs will have “prologues” and “epilogues” from now on, “headers” and “footers” are so passe :-)

Equal to the presentations was the social side of things too. I do enjoy hanging around in pubs and talking shop.

DOAG 2014 review

I had merely 10 hours at home between the end of the 2014 OTN APAC tour and my travel to Nuremberg for this year’s DOAG conference which I picked over the last two OTNAPAC stops in Bangkok and Auckland even though I am sure the weather there would have been much better than what Nuremberg had to offer.

the pragmas at DOAG2014But the DOAG conference is always worth a visit and has constantly been improving over the last few years. It has seen a huge increase in foreign speakers that elavated the level of the content to one of the best conferences worldwide. And it also seems to me that my fellow germans have warmed up much more to the general idea of networking and easy chatting.

Unfortunately most of my conference experience this year was dedicated to “slide polishing” in a quiet corner. They accepted two brand new talks and both slidedecks were in need of some extra last-minute work. In my first presentation I gave an introduction to puppet config management for Linux and Solaris and a bit of a taste of how to enable devops for Oracle databases and managing resources within DBs with a puppet module developed by Bert Hajee. The talk went very well and I was surprised by the rather large audience for a systems topic. Yay!

My second talk was due on the last day of the conference, and I was going to speak about open source and CLI tuning tools for the database. This was scheduled on the big stage in room “Tokyo”. There were bright lights, a simultaneous translator who struggled to keep up with my fast-paced german and quite an audience. I felt like I had so much to cover during this talk that I sped through the first half way too quickly when really I should have spent a lot more time with the demos. In the end, I finished after 30 minutes with 15 minutes to spare in my timeslot. The general feedback seemed to be that it was “very energetic” and noone seemed to be bitter to get to lunch 15 minutes too early. But I will have to work on this issue for my next presentation.

DOAG2014 RACAttack and CloneAttackAfter a big success in 2012 and a pause in 2013 it was time again for RACattack. And this time we also brought along RepAttack with the help of Dbvisit and Dell aswell as CloneAttack run by Kyle Hailey. Attendees had the chance to get their hands dirty on their own laptops and install a 12c RAC cluster, setup logical replication or their own Delphix database clone environment under the guidance of true experts. Unfortunately, the attendance was lower than we had hoped and experienced at past workshows, so either the content of the conference was just too good to miss or we did not properly get the message out. In any case, if you have suggestions on how to improve these hands-on labs, please let us know.

OTN APAC tour 2014 blog posts compilation

So much has been written about all the exciting events of the 2014 OTN Asia-Pacific tour. It just feels right to compile all of them into one link list. I am sorry if I missed a post, please ping me and I promise I will add and fix that.

General posts before the tour

singapore supertrees and MBS from satay by the bay bathroomStop 0 – Singapore

Stop 1 – Perth, Australia

OTN APAC Shanghai posterStop 2 – Shanghai, China

Stop 3 – Tokyo, Japan

OOW swim beach balls in BeiijingStop 4 – Beijing, China

Stop 5 – Bangkok, Thailand

Stop 6 – Auckland, New Zealand

post tour

OTN APAC tour stop 4 – Beijing, China

Beijing Olympic Green fenced off for APECAfter arriving and resting for all afternoon, I headed out to the one sight that I was keen on seeing in Beijing. Our hotel was part of the infrastructure built for the 2008 Olympics and the birds nest and aquatic centre were just one block away. So I had planned to go there and also swim some laps in the same pool where Michael Phelps set a world record. But that was not going to happen. I learned that the whole olympic green area was still blocked after the APEC summit which concluded just a few days before our arrival. And it remainded closed during our whole time in Beijing. The upside to this was that we were treated with relatively little air pollution as according to this piece by Reuters the Chinese shut down lots of factories before and during the APEC summit and even stopped giving out marriage licenses in order to present itself in the best light.

ACOUG 2014 keynoteThe conference started early the next morning and despite me not understanding chinese, I still went to see a bit of the opening keynote. They had a massive stage in a giant ballroom and most of the 1000 seats were taken with many people standing in the back. It was a very impressive setup and it looks like they shared a lot of good news with their members. I was also surprised to hear a bit of english when they played a video message by IOUG president John Matelski.

After lunch they seperated the keynote hall into two rooms that were still of a significant size. Tom Kyte filled one of them easily and Tim also had a good turnout for his PL/SQL specific session. After this, I quickly took a taxi downtown to take a few selfies in the forbidden city and came back in time for the last session and the speakers dinner. Giuseppe was back after his first appearance on the tour in Shanghai and Joel Perez joined on this second half of the tour and will do the last legs in Bangkok and Auckland. Joel is always full of stories about travel and life in general.

speaking at ACOUG OTC 2014I was up to speak on the second day of the conference in the same timeslot as Tom Kyte and Giuseppe. Not fair! But it turned out really well, my large room was about half full and Giuseppe reported that there was standing room only in his talk at the mysql track. The organizers warned me multiple times to not touch the LED screen for risk of getting electrocuted and I made sure to keep a safe distance.

Even though I did not really attend any of the sessions since I have either seen them already or would not understand chinese, there were plenty of things to do. There was a good number of vendors to see and chat with. And at the far end of the hall they set up a media centre for video interviews and a photo studio where I had some make up put on and tons of photos shot. They arranged my hair in a very weird way but I was not going to start an argument here. Will post pictures as soon as they get to me.

OOW swim beach balls in BeiijingWhich brings me to my favourite part: photo time with the locals. There was a bunch of young DBAs that was having their shirts signed by all the foreign speakers and were keen to pose for pictures. Silly faces were made, thumbs were put up and in the end we exchanged wechat contacts. Since the great firewall of china makes most western social media platforms painful to use (read: blocked), wechat is very popular in China. I had never even heard about it before but downloaded the app earlier that day. It is actually quite impressive. Everyone in China seems to be on it with people and businesses putting theit contact info on business cards, restaurant menus, bus stations, etc. Also, people actually use QR codes a lot to add contacts.

signing t-shirts in BeijingAfter a quiet and easy dinner with Tim and Joel it was time to say early goodbyes since they had to take another early flight to their next destination. For me, this marked the end of this year’s OTN APAC tour as I will be travelling home for one night and then on to the DOAG conference right after that. I always get a bit sentimental at the end of conferences and tours but just need to remind myself how lucky I am to be able to keep doing this, meet friends from all over the world again and still make new ones at every location I get to visit. And so I am leaving Asia overwhelmed with new impressions and honoured to have made many acquaintances and new friends. I hope I will be back rather sooner than later.

OTN APAC tour stop 3 – Tokyo, Japan

I woke up on Tuesday Morning with a bit of shock on confusion. Where was I? Did I oversleep? What day is it? Do I have to be at the airport and fly somewhere again? When you travel to a new destination every two to three days with jetlag, red-eye flights and all sorts of new impressions to digest, things start to get blurry. Tim and I had arrived the night before from Shanghai and even though the flight itself was just a quick 2,5 hours jump over the Sea of Japan, we were kept busy with queueing and rides in taxis and trains.

Shinagawa stationAfter a shower and some toying around with the japanese high-tech toilet (there were LEDs and buttons to explore) I got on my way to the conference location using the public transit system that I had researched way ahead of time. The google maps app has been a great help with this, too. When searching for public transit connections, it not only shows the line you need to take but also the ticket price and the number and names of the stations in between. Back in Singapore I was already impressed by the ability of the app to display information on the different floors in an indoor shopping mall by floor.

But none of the reading I did really prepared me for the masses of people that rushed through those stations on their morning commute. With a new train arriving from each direction every 3 minutes, there was an endless stream of people. But even though the sheer number was mindblowing there was no chaos at all. It was all very civilized and orderly with people respecting each others privacy and there was no shoving or body contact of any kind. It was very fascinating to watch which made me the only person not following the stream as I was mostly standing there in awe.

Francisco Noriko BjoernThe conference center was conveniently located right next to the Akihabara train station and easy to find. They had four rooms with about 60 seats each which is a great size. A huge ballroom can sometimes be intimidating (when it is full) or deserted (when there are too many empty seats), and this was just right. Insight technology, the organizer and main sponsor of the db tech showcase conference, did an outstanding job. There were coordinators in bright yellow t-shirts everywhere and every possible detail was covered. Sessions started right on time with the strike of the clock, they recorded all the sessions and made sure that speakers and delegates were looked after. A photographer captured some great shots and was quick to upload loads of picture to social media sites. Our liaison Noriko deserves a special mention. She seemed to always be hovering around, making sure that we always had everything we needed, got us fed during lunch, sang and danced with us and was just a sheer bundle of joy.

presenting in TokyoCraig started the conference with an excellent introduction and explanation of the working of latches and mutexes in his typically captive style. I was due to present two presentations on the first day and despite being a bit nervous about how the japanese audience would enjoy my talks I was very happy with my delivery. But it took a bit of getting used to how the Japanese react (or don’t) to the presenters. They showed very little reaction and only a very occasional nod or smile and did not raise questions during or right after the talk. But to my relieve I was able to speak to a handful of the delegates one-to-one at the reception that night and was reaffirmed that they honestly liked all of our presentations, so I reckon not speaking out or showing reactions in public is just a sign of japanese politeness.

Craig and Tim at lunchIt was also good to see Francisco again, and he talked about the challenges organizations are facing when implementing (or even planning to do so) high availability and desaster recovery solutions. He showed some very interesting (yet disturbing) statistics he gathered from a user group survey and showed ways to get started and take action. One of the last tips he gave resonated very well with me because I had the same thought for a long time but was wondering why noone actually uses this: He mentioned that when you do replication it is sensible to keep two replicated databases. One that is synced almost synchronously as the pure (physical) DR database and another one that is intentionally set to lag behind by a day or two. That way, you have an easy starting point in case you need to perform a back in time recovery. So you only have to recover and apply the latest archive logs instead of having to go to your backups and perform a lenghty restore operation. Tim did his talk about avoiding PL/SQL performance mistakes again and unfortunately I could not really attend (or understand) the other sessions since everything else was in japanese.

And I also met Anton from New Zealand again who must have been the only one from his team that did not make it to OpenWorld this year. So it was good to reconnect but a shame that he was not presenting until Thursday which was also the day when we were flying out to Beijing.

bingo with Hiroki and MisaThere was a networking party later on that first night and it did live up to the conference motto “meet wow”. The staff all dressed up in cute christmas costumes of santas, reindeer, snowmen and a christmas tree while playing bingo with the attendees. This (and beer) helped to break the ice and I got to talk to some of the other attendees of the conference. The consensus here: no matter where you live or what your cultural background is: DBAs are struggling with the same issues everywhere; Oracle bugs (and features) do not discriminate.

symphony no 5The highlight of the night was a performance of an all-girl J-pop band called Symphony No 5. This group of 8 girls in manga-inspired costumes, high-pitch voices and perfect choreographies captured the audience that was swinging glow-sticks with the rhythm (I know that does not sound right). I thought it was brilliant and I loved it.

The second day was packed with great content by Tom Kyte, Mike Dietrich, Craig and Tim. But since I already did two talks on the first day, I had some spare time to explore Akihabara after lunch. This district is heaven on earth for geeks. An area of several blocks has all the electronics, video games, arcades, anime and manga shops that you could ever want. A friend of mine is into a few of the anime shows which gave me an excuse to hunt for some memorabilia in a few of the malls. My research and advice from the locals also advised that theme cafes are a huge trend in the area. These are places where the waiters dress up in costumes – usually maids – and interact with the customer by playing card games, singing songs together and drawing cute cat faces with chocolate syrup on desert dishes. I had to go and see this for myself. They gave me cat ears and I learned a few “cute” words and gestures, it was a bizarre experience. But the weirdest part were the customers. Sure, there were a couple of tourists but even more a good number of local regulars with frequent-maiden cards who were greeted with their name and they brought a huge collection of pictures and autographs.

OTN APAC team in JapanLater we were treated to a very nice dinner with the organizers. Steamed vegetables and crab meat, lots of fish, scallops and clams, it was a whole new world to taste. And it was also a good chance to chat with the locals and exchange stories and jokes which to me is the best part about these OTN tours. What better way to learn about a foreign place than to spend time with locals. You start a connection through technology and soon become friends that talk about anything but technology. I was honored to also join the group for some karaoke after the dinner. Again, what impressed me most was how big the Japanese are into this. The place we went to consisted of three stories and each one had a number of private cabins that you could rent as a group. When we got there, the doors to a few of those were open and I got to take some peeks inside and see the crazy parties some were having. We mixed it up between japanese and english songs, I got to sing “Big in Japan” and we finished the night with all of us singing “Hey Jude” by the Beatles which reminded me that I would be in Liverpool for UKOUG in just three weeks.

After this it was time to say goodbye to my newfound friends who still had one more day of conference ahead of me while Tim and I would have to get up in the middle of the night to catch a flight to Beijing. But we would be treated to sunrise over Tokyo Bay on the way to the airport.

ORA-64359: INMEMORY clause may not be specified for virtual columns

The title above gives you immediate knowledge of the end result of this post. If you want to know the whys and wherefores then by all means read on.

I was on a call yesterday listening to a presentation on Database 12c new features. The In-Memory Column Store received a lot of interest, as did the new JSON functionality.

One person on the call asked an interesting question, it was something along the lines of:

“If we have a table containing a JSON document can we use the In-Memory Column Store to optimise reports on the JSON attributes”

After some discussion it was decided that the best way to do that is to store the required JSON attributes in dedicated regular columns and report on those. After the call I thought about this some more and wondered if we can:

a) Expose JSON attributes as virtual columns and…
b) Utilise the In-Memory Column Store to report on those virtual columns

I thought these tests were worth executing. Here they are:

a) Can we expose JSON attributes as virtual columns

In order to set up the test I create a table with a JSON column and a virtual column using the function JSON_VALUE to retrieve a scalar value for an attribute called “Name”.

(   id          NUMBER (10) NOT NULL
,   date_loaded DATE
,   doc         CLOB
    CONSTRAINT ensure_json CHECK (doc IS JSON)
,   doc_name AS (JSON_VALUE(doc,'$.Name'))

Table created.

SQL> desc json_docs
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                                        NOT NULL NUMBER(10)
 DATE_LOADED                                        DATE
 DOC                                                CLOB
 DOC_NAME                                           VARCHAR2(4000)

Looks promising. Let’s add some data.

insert into json_docs (id, date_loaded, doc)
values (1,sysdate,'{"Name": "Neil", "Email": "neil@email.com", "Job": "DBA"}');
insert into json_docs (id, date_loaded, doc)
values (2,sysdate,'{"name": "Bob", "Email": "bobl@email.com", "Job": "DBA"}');

set lines 120
column doc format a60
column doc_name format a10
select * from json_docs;

 ID DATE_LOAD DOC                                                          DOC_NAME
--- --------- ------------------------------------------------------------ --------
  1 13-NOV-14 {"Name": "Neil", "Email": "neil@email.com", "Job": "DBA"}    Neil
  2 13-NOV-14 {"name": "Bob", "Email": "bobl@email.com", "Job": "DBA"}

And there we have it. When the attribute “Name” is present at the top level of the JSON document then we can show it as a dedicated virtual column. I’m not suggesting this is a good idea but… it works.

b) Can we utilise the In-Memory Column Store to report on virtual columns

First I’ll add more data to the JSON_DOCS table.

delete json_docs;
insert into json_docs (id,date_loaded,doc)
select rownum, sysdate, '{"Name": "'||object_name||'", "Type": "'||object_type||'"}'
from all_objects
where rownum <= 500;

And set the table to be INMEMORY, excluding the JSON document column.

SQL> show parameter inmemory_size

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
inmemory_size                        big integer 252M


Next query the data to kick of a load to the “In-Memory Area”

select count(*) from json_docs;
column segment_name format a30
select segment_name,populate_status,bytes_not_populated from v$im_segments order by 1;

------------------------------ --------- -------------------
JSON_DOCS                      COMPLETED                   0

That looks promising. So let’s check which columns are cached.


--------------- ---------------- ---------------------
JSON_DOCS       ID               DEFAULT
JSON_DOCS       DOC              NO INMEMORY

Not the virtual column… Can we force it:

ALTER TABLE json_docs INMEMORY (date_loaded, doc_name) NO INMEMORY (doc) ;

ERROR at line 1:
ORA-64359: INMEMORY clause may not be specified for virtual columns

Nope. And probably for good reasons.

OTN APAC tour stop 2 – Shanghai, China

Contrary to what my family, friends and coworkers believe, not every aspect of these OTN tours is glamour or relaxation, but there is actually quite a lot of dull travelling involved (with the time used to compose blog posts like this one). The travel to Shanghai was such a day. After a last dinner in Perth, Tim and I hitched a ride with Connor to the airport, then spent the first part of the night there, waiting for our plane to depart shortly after 2am. I managed to sleep a fair bit of that first leg to Singapore where the little transition time that we had was a great reminder of the wonderful time I spent there just the prior weekend. This was one of those times were I would not have minded a cancelled flight and an extra day there. But our flight to Shanghai was right on time and another 4 to 5 hours long.
We took the maglev train from the airport which was quite an experience because a) you do not get to ride 430km/h every day and b) it was good to see what was done with my german tax money that went into the research of this technology. I felt adventurous so instead of taking a taxi for the final bit to the hotel we took the metro after a bit of a fight with the ticket machine. First of all there was only one machine that accepted bills so it had a bit of a queue. And when it was our turn we could not buy the tickets we needed because the english screen did not allow us to select the right line (only a subset). But after a bit of pointing the guy behind us helped out.

OTN APAC Shanghai posterLater I enjoyed a wonderful dinner in a restaurant that was also partly a museum, met two of the other speakers. Maclean Liu from the UG and also the sponsororing partner in Shanghai, Joe Huang who has been working for Oracle at HQ as a product manager for years but was raised in Taiwan, so he knew mandarin. And then also our liasion and local coordinator Bo Feng who did an excellent job at looking after the international speakers and introduced us to a lot of new and strange food items.

Again, I felt adventurous and walked from my hotel to the conference site which was about 40 minutes. And I was pretty much blown away by stepping into a completely different world. I passed a street packed with little hardware shops selling all kinds of screws, nuts and bolts and other building materials, then some very local and raw parts of town next to the river. It really felt like diving into a completely different world.

The conference started with updates by 刘冰冰, a lady from Oracle, and even though all I could understand were a couple of keywords (CDB, PDB, big data SQL, cloud) I am pretty sure I have heard a very similar presentation in English a few times already. The audience understood english well enough but the organizers also translated the slides of Tim and Giusseppe Maxia into chinese so when they were talking they sometimes had to guess at what each bullet point said because there was not much english text left. Tim talked about how to avoid the most commin performance problems in PL/SQL and it was an excellent presentation with good points and a lot of small demos.
Giuseppe talked about mysql replication, how to set it up and also how to monitoring and some advanced cases, I am already looking forward to seeing him again at the Beijing conference in a few days.

OTN APAC Shanghai raffle winnersI did my talk on RAC connectivity which I have done many times now. I am sometimes getting a bit tired of my older presentations but still like this one a lot and think I should maybe turn this more into a general talk about RAC rather than just talk about load balancing and services.

Since the rest of the sessions were going to be in chinese we snuck out and walked around a bit. One of the more bizarre things we saw was a shopping mall with 3 or 4 floors of nothing but mobile phones and accessories. Even some repair shops were people were taking phones apart. We wondered how many of the phones we saw were counterfeits and how many were originals.

OTN APAC Shanghai selfieWe made it back to for the end of the conference and enjoyed another great dinner with the guys from the Shanghai User Group and speakers. The oddest food was fermented stinky tofu. Think of blue cheese. Tim made the mistake of taking a sniff first, then eating, I just avoided breathing through my nose. The taste was actually pretty good.

I decided to go for a brief walk over to the Bund and take some pictures of the Pudong skyline. It was a bit over the top with the colourful blinking lights and screens and couples in cheap tuxedos and polyester dresses having their pictures taken. The constant presence of hawkers advertising cheap “massages” was very annoying and I dared to walk about in some of the sidestreets and was rewarded with scenes of street food carts, interesting smells (both good and bad) and just generally loads of bustling life happening in the streets.