Quantcast
Channel: Gokhan Atil’s Technology Blog
Viewing all 163 articles
Browse latest View live

New Oracle ACE Director: Me :-)

$
0
0

sampanya-500x305

I’m very proud to announce that I am accepted as an Oracle ACE Director. Now I’m part of a special club which has only 133 members (for now) in the world. It’s surely a milestone for my carrier. I would like to thank to Gurcan Orhan, Kellyn Pot’Vin-Gorman, Jeff Smith, Julian Dontcheff, Kamran Aghayev and Heli Helskyaho for being reference for my ACED nomination. I would also like to thank Oracle ACE program and committee members that approved my nomination.

I know that Oracle ACE Director title will put more responsibility to my shoulders. I have to be more active in the Oracle community in future. I have accepted the challenge and I’ll do my best!

I received another good news today for Turkish Oracle User Community. TROUG board member, Oracle Database Developer Choice Awards SQL 2015 Winner, Emrah Mete is accepted to Oracle ACE program. Congratulations Emrah!


OMS Upgrade Fails At Repository Configuration With Error ORA-20251

$
0
0

Yesterday, one of my blog readers asked me how I didn’t encounter ORA-20251 error when upgrading my EM12c to EM13c. He also pointed My Oracle Support document (Doc ID 2095025.1). The document says a new record will be inserted into EM_COST_CENTERS table (in fact the table name is EM_COST_CENTERS_E but it’s not relevant) for all users whose department value is not blank during the upgrade. If there are more than one records with blank cost center then the second record will be considered as duplicate, resulting in the above error during the repository configuration.

Here’s the sample output:

INFO: oracle.sysman.top.oms:Failure…
INFO: oracle.sysman.top.oms:1 error(s) happened in performing the action:
INFO: oracle.sysman.top.oms:1):
ORA-20251: Cost Center already Exists : :
ORA-06512: at “SYSMAN.EM_COST_CENTER”, line 147
ORA-06512: at “SYSMAN.EM_COST_CENTER”, line 209
ORA-06512: at “SYSMAN.EM_COST_CENTER”, line 266
ORA-06512: at “SYSMAN.EM_COST_CENTER”, line 436
ORA-06512: at line 64
File:…/cost_center/cost_center_data_upgrade.sql

I asked my reader to examine the EM users and he said there are not any EM users whose department value is not blank. So today, I examined the upgrade script and see that it’s not just about the department value. The upgrade scrip also tries to insert new “cost center” records for the users whose Line of business value is not blank.

Here’s the list of my EM users:

emcli list -resource="Administrators" \
 -columns="USER_NAME,DEPARTMENT,LINE_OF_BUSINESS,COST_CENTER"
USER_NAME             DEPARTMENT            LINE_OF_BUSINESS      COST_CENTER
SYSMAN
GOKHAN                                                            COSTCENTER1
CLOUD_SWLIB_USER
VEYSEL                DENEME1               DENEME2
NURFIDAN                                                          COSTCENTER2
SUPPORT
Rows:6

As you can see, there’s only one user whose department and line of business values are not blank. The upgrade script (cost_center_data_upgrade.sql) will create a new cost center for the department of this user, and there won’t be any error. On the other hand, if there were another user whose cost center value is blank but department or line of business value is not blank, the script would try to create another cost center and it would cause a duplicate record violating unique constraint (COST_CENTER_NAME, TENANT_ID) on EM_COST_CENTERS_E table.

So before you upgrade the EM13c, you better check your users with the following command (I break the line so it can fit to the page, you can write it as one-line):

emcli list -resource="Administrators" \
-columns="USER_NAME,DEPARTMENT,LINE_OF_BUSINESS,COST_CENTER"

If there are multiple users with department or line of business value but not assigned to a cost center, you will hit the bug. Unfortunately, the My Oracle Support document doesn’t provide clear instructions to fix the problem. I recommend you to contact support and ask them to update the document (Doc ID 2095025.1). Meanwhile, if you feel adventurous, you may use a script similar to the below one, to set department value (or line of business value) to cost center value:

login()

em_users = list( resource="Administrators",
columns="USER_NAME,DEPARTMENT,LINE_OF_BUSINESS,COST_CENTER",
search="COST_CENTER IS NULL")

for em_user in em_users.out()['data']:
  if em_user['DEPARTMENT'] is not None:
    modify_user( name=em_user['USER_NAME'], cost_center=em_user['DEPARTMENT'] )
  elif em_user['LINE_OF_BUSINESS'] is not None:
    modify_user( name=em_user['USER_NAME'], cost_center=em_user['LINE_OF_BUSINESS'] )

Before executing the script:

emcli list -resource="Administrators" \
-columns="USER_NAME,DEPARTMENT,LINE_OF_BUSINESS,COST_CENTER"
USER_NAME             DEPARTMENT            LINE_OF_BUSINESS      COST_CENTER
SYSMAN
GOKHAN                DENEME1
CLOUD_SWLIB_USER
VEYSEL                                      DENEME2
NURFIDAN              DENEME3
SUPPORT
Rows:6

After the executing the script:

emcli list -resource="Administrators" \
-columns="USER_NAME,DEPARTMENT,LINE_OF_BUSINESS,COST_CENTER"
USER_NAME             DEPARTMENT            LINE_OF_BUSINESS      COST_CENTER
SYSMAN
GOKHAN                DENEME1                                     DENEME1
CLOUD_SWLIB_USER
VEYSEL                                      DENEME2               DENEME2
NURFIDAN              DENEME3                                     DENEME3
SUPPORT
Rows:6

After assigning cost centers to the users, the upgrade script will work without getting any error.

How to Install Oracle 11gR2 XE on Ubuntu

$
0
0

I wanted to import a datapump dumpfile to my laptop, and I wondered if I can install Oracle XE to ubuntu guest. I thought it would be easy because I installed Oracle 10g XE to ubuntu for several times, but I saw that Oracle doesn’t provide debian packages for Oracle 11g XE. So I needed to convert the RPM to debian package (and fix incompatibility problems). Therefor, I installed “alien” package and other required tools using apt-get:

gokhan@gokhan-VirtualBox:~/Desktop$ sudo su -
root@gokhan-VirtualBox:~# apt-get install alien

Then I downloaded the zipped RPM (oracle-xe-11.2.0-1.0.x86_64.rpm.zip), unzipped it:

root@gokhan-VirtualBox:~# unzip oracle-xe-11.2.0-1.0.x86_64.rpm.zip
root@gokhan-VirtualBox:~# cd Disk1
root@gokhan-VirtualBox:~/Disk1# ls -l
total 309892
-rw-rw-r-- 1 root root 317320273 Ağu 28  2011 oracle-xe-11.2.0-1.0.x86_64.rpm
drwxr-xr-x 2 root root      4096 Ağu 28  2011 response
drwxrwxr-x 2 root root      4096 Ağu 28  2011 upgrade

I converted the RPM to debian package (it took about 6 minutes on my VM) and then installed the package (which I created in previous step):

root@gokhan-VirtualBox:~/Disk1# alien --scripts oracle-xe-11.2.0-1.0.x86_64.rpm 
oracle-xe_11.2.0-2_amd64.deb generated
root@gokhan-VirtualBox:~/Disk1# dpkg -i oracle-xe_11.2.0-2_amd64.deb 
Selecting previously unselected package oracle-xe.
(Reading database ... 236797 files and directories currently installed.)
Preparing to unpack oracle-xe_11.2.0-2_amd64.deb ...
Unpacking oracle-xe (11.2.0-2) ...
Setting up oracle-xe (11.2.0-2) ...
Executing post-install steps...
/var/lib/dpkg/info/oracle-xe.postinst: line 114: /sbin/chkconfig: No such file or directory
You must run '/etc/init.d/oracle-xe configure' as the root user to configure the database.

Processing triggers for ureadahead (0.100.0-16) ...
ureadahead will be reprofiled on next reboot
Processing triggers for gnome-menus (3.10.1-0ubuntu2) ...
Processing triggers for desktop-file-utils (0.22-1ubuntu1) ...
Processing triggers for bamfdaemon (0.5.1+14.04.20140409-0ubuntu1) ...
Rebuilding /usr/share/applications/bamf-2.index...
Processing triggers for mime-support (3.54ubuntu1.1) ...
Processing triggers for libc-bin (2.19-0ubuntu6.7) ...

As you can see, the installation script tried to use “/sbin/chkconfig” but it couldn’t find it. On the other hand, there was not any big problems occurred, so I decided to continue with configuration:

root@gokhan-VirtualBox:~# /etc/init.d/oracle-xe configure 

Oracle Database 11g Express Edition Configuration
-------------------------------------------------
This will configure on-boot properties of Oracle Database 11g Express 
Edition.  The following questions will determine whether the database should 
be starting upon system boot, the ports it will use, and the passwords that 
will be used for database accounts.  Press <Enter> to accept the defaults. 
Ctrl-C will abort.

Specify the HTTP port that will be used for Oracle Application Express [8080]:

Specify a port that will be used for the database listener [1521]:

/etc/init.d/oracle-xe: line 405: /bin/awk: No such file or directory
Specify a password to be used for database accounts.  Note that the same
password will be used for SYS and SYSTEM.  Oracle recommends the use of 
different passwords for each database account.  This can be done after 
initial configuration:
Confirm the password:

Do you want Oracle Database 11g Express Edition to be started on boot (y/n) [y]:

Starting Oracle Net Listener...touch: cannot touch '/var/lock/subsys/listener': No such file or directory
Done
Configuring database...
Database Configuration failed.  Look into /u01/app/oracle/product/11.2.0/xe/config/log for details

Although I just entered default values, the configuration script failed. On line 15, I see that it tries to use /bin/awk – ubuntu’s awk is located at /usr/bin/awk. On line 24, it tries to create a listener “lock” file in “/var/lock/subsys/” and when I check the directories, I see that there’s no subsys folder under the “/var/lock”. So I create symbolic link for awk and create that “subsys” folder:

root@gokhan-VirtualBox:~# ln -s /usr/bin/awk /bin/awk
root@gokhan-VirtualBox:~# mkdir /var/lock/subsys

Then I examine the logs in “/u01/app/oracle/product/11.2.0/xe/config/log”, I see that database couldn’t be created, and the reason seems “ORA-00841: MEMORY_TARGET not supported on this system”. It typically occurs when “/dev/shm” was not sized correctly on Linux. On Ubuntu, the /dev/shm is just a symbolic link which points to /run/shm. So I removed the link, recreated the directory and mount 1 GB RAM to it:

root@gokhan-VirtualBox:~# rm /dev/shm
root@gokhan-VirtualBox:~# mkdir /dev/shm
root@gokhan-VirtualBox:~# mount -t tmpfs shmfs -o size=1024m /dev/shm

After setting the /dev/shm, I tried again to configure the Oracle XE and it worked! Then I switched to oracle user, and connected to Oracle XE database:

oracle@gokhan-VirtualBox:~$ export PATH=product/11.2.0/xe/bin:$PATH
oracle@gokhan-VirtualBox:~$ . oraenv
ORACLE_SID = [oracle] ? XE
ORACLE_BASE environment variable is not being set since this
information is not available for the current user ID oracle.
You can set ORACLE_BASE manually if it is required.
oracle@gokhan-VirtualBox:~$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Fri Apr 8 10:47:12 2016

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production

SQL>

I wanted to share how I fixed the problems when I encounter installing Oracle XE on Ubuntu, so I just created an empty Ubuntu guess, and re-apply all commands to make sure that I don’t miss anything. Guess what! Somehow it didn’t worked. I can surely swear that I exactly did the same things then I remembered that I installed extra packages for different purposes and some of them must be used by Oracle XE.

I checked the library usage of oracle executable and see that “libaio.so.1” is missing. So I also installed “libaio1” package:

apt-get install libaio1

It solved the problem but I needed to start from scratch (purged the package and then re-install, re-configure) and everything seems OK for now. If you reboot your VM, you’ll see that /dev/shm will be reverted to symbolic link and the subsys folder is gone. It might be possible to modify Ubuntu’s configuration scripts to prevent this behaviour. On the other hand I recommend you to create a simple bash script to mount /dev/shm and startup oracle-xe, instead of modifying Ubuntu’s configuration files. For example:

root@gokhan-VirtualBox:~# cat startxe.sh 
rm /dev/shm
mkdir /dev/shm
mkdir /var/lock/subsys
mount -t tmpfs shmfs -o size=1024m /dev/shm
/etc/init.d/oracle-xe start

TROUG Days 2016 İstanbul Twitter Statistics

$
0
0

TROUG Days İstanbul was held last week (April 28th to 29th) at İstanbul Technical University Technopark. I think we used social media, specially twitter very effective, so I wanted to publish some twitter statistics about the event like I did for OUGN Spring Event. The statistics cover the period between April 27 and April 30.

Before sharing the twitter statistics, I would like to write a few words about the event. I feel very proud to be a part of the organization team of TROUG Days İstanbul. It was the first two-day event organized by TROUG and everything went OK. It is a milestone for TROUG. We increased the expectations of TROUG members, and will surely organize a better one next year. I would like to thank our sponsors (Oracle Turkey, Axxana, Bilginç IT Academy) and all speakers. We had a great lineup including 10 Oracle ACE Directors, 7 Oracle ACEs, 3 Oracle ACE Associates.

When I say “speakers”, I remembered that we had a great speakers’ dinner thanks to ACE Program and our sponsors. I hope the speakers liked our hospitality and they will come again next year! After this brief summary of the event, here are the twitter statistics:

trougtwitter

#TROUGDays Tweets

Number of tweets (including RTs):

942

Most Favourited Tweets:

Uwe Hesse @UweHesse
#TROUGDays @brost presenting about free Oracle Tuning Tools https://t.co/Cj1lSMjO0V
Thu Apr 28 10:32:13 +0000 2016 retweeted 14 favourited 18
Uwe Hesse @UweHesse
Join me 16:30 in Hall A presenting “Best of RMAN” at #TROUGDays with many live demonstrations 🙂 #Oracle https://t.co/EeuKP6hNek
Thu Apr 28 12:39:35 +0000 2016 retweeted 8 favourited 17
Gurcan Orhan @gurcan_orhan
Having fun during the break. @oracle_ckpt @gokhanatil @OsamaOracle @Ycolin #TROUGDays @oracleace @OracleTR https://t.co/mZk4r6y6Sj
Thu Apr 28 09:14:05 +0000 2016 retweeted 7 favourited 17
Gokhan Atil @gokhanatil
#TROUGDays @oracleace Speakers Dinner… Things are getting hot! 🙂 https://t.co/QguoAGkA8B
Thu Apr 28 16:26:43 +0000 2016 retweeted 6 favourited 17
Gokhan Atil @gokhanatil
Listening @OsamaOracle speaking “Weblogic 101 for DBAs” at #TROUGDays. https://t.co/kxqBnhqLod
Thu Apr 28 11:42:06 +0000 2016 retweeted 10 favourited 16

Most Retweeted Tweets:

Uwe Hesse @UweHesse
#TROUGDays @brost presenting about free Oracle Tuning Tools https://t.co/Cj1lSMjO0V
Thu Apr 28 10:32:13 +0000 2016 retweeted 14 favourited 18
Gokhan Atil @gokhanatil
#TROUGDays has started with introduction speech of @gurcan_orhan. Now @yasinbaskan is on the stage. https://t.co/i6Fn3fUNva
Thu Apr 28 06:45:16 +0000 2016 retweeted 13 favourited 12
TROUG @TurkishOUG
Last preparations are on the way. Badges are ready. #TROUGDays https://t.co/aemna2OJKu
Wed Apr 27 11:10:01 +0000 2016 retweeted 12 favourited 8
Uwe Hesse @UweHesse
Turkish Oracle User Group Conference in Istanbul 2016 #TROUGDays https://t.co/1dRAh6Zp3d https://t.co/4RizGohXyF
Fri Apr 29 07:32:05 +0000 2016 retweeted 11 favourited 15
TROUG @TurkishOUG
Please note that TROUG Days 2016 official Twitter hashtag is : #TROUGDays
Wed Apr 27 11:08:56 +0000 2016 retweeted 11 favourited 5

Most Visible Tweets:

Uwe Hesse @UweHesse
#TROUGDays @brost presenting about free Oracle Tuning Tools https://t.co/Cj1lSMjO0V
Thu Apr 28 10:32:13 +0000 2016 retweeted 14 favourited 18
Gokhan Atil @gokhanatil
Listening @OsamaOracle speaking “Weblogic 101 for DBAs” at #TROUGDays. https://t.co/kxqBnhqLod
Thu Apr 28 11:42:06 +0000 2016 retweeted 10 favourited 16
Uwe Hesse @UweHesse
Turkish Oracle User Group Conference in Istanbul 2016 #TROUGDays https://t.co/1dRAh6Zp3d https://t.co/4RizGohXyF
Fri Apr 29 07:32:05 +0000 2016 retweeted 11 favourited 15
Gokhan Atil @gokhanatil
#TROUGDays has started with introduction speech of @gurcan_orhan. Now @yasinbaskan is on the stage. https://t.co/i6Fn3fUNva
Thu Apr 28 06:45:16 +0000 2016 retweeted 13 favourited 12
Sten Vesterli @stenvesterli
Summary of my #TROUGDays presentation. See https://t.co/SOltqFK9gO for more https://t.co/tf2Q2CnjC3
Thu Apr 28 12:05:13 +0000 2016 retweeted 8 favourited 8

Most Active Twitter Users:

Tweets
TROUG @TurkishOUG
Türk Oracle Kullanıcıları Grubu | Turkish Oracle User Group official Twitter account.

49

Mahir M. Quluzade @marzade
Oracle ACE, OCP/OCE DBA/Developer. Board Member of Azerbaijan Oracle User Group,Speaker, Author, Blogger

19

emrah topcu @emrah_topcu
Product Manager

17

Gurcan Orhan @gurcan_orhan
Awarded (2011-Enterprise Architect) Oracle Data Integrator guy, Oracle #ACED, DWH Architect, blogger, blues guitar player, married, father. Views are own.

16

Mustafa KATRANCI @mustikatranci
Senior Technical Consultant, Oracle ADF Developer, FMW Team Lead. PL/SQL lover. Board member of TROUG. Father, husband, BeşiktaŞk, photographer, blogger

13

About the Lists:

  • The twitter data was fetched on May 1st 2016. So the retweeted and favourited counts might have been changed.
  • Most favourited tweets are sorted based on favourited count (descending), retweeted count (descending) and tweet date.
  • Most retweeted tweets are sorted based on retweeted count (descending), favourited count (descending) and tweet date.
  • Most visible tweets are calculated based on the followers count of the original tweeter and retweeters.
  • Most active twitter users are calculated based on the total number of tweets and retweets.

OUGF Harmony 16 Spring Conference

$
0
0

Last week I took part in the OUGF (Oracle User Group Finland) Harmony 16 Spring Conference. It was a two day event, and held in Helsinki, between May 18th and 19th. In the conference, I gave two speeches: “Essential Linux commands for DBAs” and “Enterprise Manager 13c for DBAs”. Most people expect to me to speak about Enterprise Manager but in my first session, I spoke about the tools which can be used for monitoring Linux performance (i.e. sar, pidstat, top), profilers and tracing tools.

ougf

In my second session, I spoke about the features of Enterprise Manager for database administrators. I think people enjoyed my sessions, because I got good questions and positive feedback from the audience. By the way, thanks Ann Sjökvist for the photo.

The Harmony conference had great sessions and most of the sessions were in English. I tried to attend to all sessions, learned a lot of good stuff, meet new people, had good conversations with my friends, so everything went great for me. I hope OUGF community also enjoyed the event as much as I do. The OUGF Management did a good job.

nassyamjulianosama

with Nassyam Basha, Julian Dontcheff, Osama Mustafa

osamasyedluis

with Osama Mustafa, Luis Marques and Syed Zaheer

pietmaris

with Piet de Visser and Maris Elsins

frank

One of the speakers, Frank Dernoncourt made a mall surprise to us, and showed he’s also a talented artist. He really plays piano well. By the way, he looks very serious in the photo, but he’s a cheerful person in real life.

roomescape

At the end of the first day of the conference, OUGF organized a dinner for attendees and speakers. We had a good meal with wine while listening nice music. After the dinner, we played “room escape” game. There were 3 groups, and each group consist of a random mixture of database administrators, developers and architects. Very talented guys to solve logical puzzles, but at the end, none of the groups escaped from the rooms 🙂 It was a very different experience and I really liked it!

Although OUGF management invited me, I missed the speakers dinner on Tuesday, but fortunately I stayed 2 more days after the conference so I found time for city sightseeing with my wife.

gokhanaylin

We visited Suomenlinna Sea Fortress, Helsinki Cathedral, Kamppi Chapel, Market Square, Senate Square and Uspenski Orthodox Cathedral. It was my first time to visit Helsinki, and I found it wonderful.

I put my presentations to the slideshare, here you can check them:

I also published a report about twitter statistics of OUGF, you can find it here: OUGF Harmony 2016 Spring Conference Twitter Statistics

Thank you OUGF! Hope to see you next year!

OUGF Harmony 2016 Spring Conference Twitter Statistics

$
0
0

If you didn’t come this page from my review about the OUGF Harmony16 Spring Conference, please don’t forget to read that one. This post is generated almost automatically by scripts to share the twitter statistics of the conference. The statistics cover the period between May 17 and May 20. I included three hashtags (#OUGF, #OUGF16, #Harmony16), because at the beginning of the conference, people (including me) did not used the official hashtag (#Harmony16).

openning


#Harmony16 Tweets

Number of tweets (including RTs):

279

Most Favourited Tweets:

Julian Dontcheff @JulianDontcheff
#OUGF @PasiJaakkola 25 years anniversary, keynote from #enkitec https://t.co/Yhop0oGxJM
Wed May 18 07:40:53 +0000 2016 retweeted 4 favourited 13
Osama Mustafa @OsamaOracle
A dinner with #harmony16 speakers thnx @HeliFromFinland @navonpilot @oracle_ckpt @Ycolin @lleturgez @drune @ora_tec https://t.co/2LNGOiCIw9
Thu May 19 16:20:26 +0000 2016 retweeted 6 favourited 11
Gokhan Atil @gokhanatil
Listening @oracle_ckpt together with @OsamaOracle @drune @ora_tec at #Harmony16 #OUGF https://t.co/eRGvWGYqiz
Thu May 19 08:27:52 +0000 2016 retweeted 3 favourited 10
Julian Dontcheff @JulianDontcheff
Uponor presenting at #OUGF on the Exadata migration performed by #accenture #enkitec and the excellent final results https://t.co/zOP4pSQQ2I
Wed May 18 07:50:40 +0000 2016 retweeted 3 favourited 10
Kamran Agayev A. @KamranAgayev
Great shot – discussing Oracle Flex ASM 12c feature with @MarisElsins having @gokhanatil on the stage #harmony16 https://t.co/DEU72RZK9V
Wed May 18 11:16:47 +0000 2016 retweeted 1 favourited 10

Most Retweeted Tweets:

Pasi Jaakkola @PasiJaakkola
@JulianDontcheff keynote at #Harmony16 #OUGF @Enkitec @AccentureTech https://t.co/QcblRy0hNp
Wed May 18 06:10:49 +0000 2016 retweeted 8 favourited 7
Pythian @Pythian
Catch @MarisElsins and @oracle_ckpt representing Pythian at #OUGF conference in Helsinki, Finland! Details at https://t.co/ggQjA7hpNQ
Tue May 17 18:51:59 +0000 2016 retweeted 7 favourited 1
Osama Mustafa @OsamaOracle
A dinner with #harmony16 speakers thnx @HeliFromFinland @navonpilot @oracle_ckpt @Ycolin @lleturgez @drune @ora_tec https://t.co/2LNGOiCIw9
Thu May 19 16:20:26 +0000 2016 retweeted 6 favourited 11
Kamran Agayev A. @KamranAgayev
@gokhanatil is on the stage talking about Oracle Enterprise Manager Cloud Control 13c #harmony16 #OUGF @TurkishOUG https://t.co/Xwe80TbKGG
Thu May 19 07:18:45 +0000 2016 retweeted 5 favourited 8
Maris Elsins @MarisElsins
I didn’t know pidstat could show disk usage per process too (-d option)! Thanks @gokhanatil #harmony16 https://t.co/acMG1eRKZP
Wed May 18 11:25:47 +0000 2016 retweeted 5 favourited 8

Most Visible Tweets:

UK Oracle User Group @UKOUG
Presenting/ attending #Harmony16 remember call for papers is open for #ukoug_tech16, submit by 1st June https://t.co/XurkfCAXHD #usergroups
Wed May 18 09:29:29 +0000 2016 retweeted 0 favourited 1
Osama Mustafa @OsamaOracle
A dinner with #harmony16 speakers thnx @HeliFromFinland @navonpilot @oracle_ckpt @Ycolin @lleturgez @drune @ora_tec https://t.co/2LNGOiCIw9
Thu May 19 16:20:26 +0000 2016 retweeted 6 favourited 11
Pythian @Pythian
Catch @MarisElsins and @oracle_ckpt representing Pythian at #OUGF conference in Helsinki, Finland! Details at https://t.co/ggQjA7hpNQ
Tue May 17 18:51:59 +0000 2016 retweeted 7 favourited 1
Pythian @Pythian
Catch @oracle_ckpt tomorrow at #OUGF Harmony Conference at 11:15 AM EEST! Event details at https://t.co/ipfpaJZTcE https://t.co/aSDRDfZXx9
Wed May 18 00:30:02 +0000 2016 retweeted 5 favourited 3
Osama Mustafa @OsamaOracle
Looking for performance issues in oracle SE by @pioro #harmony16 https://t.co/06LzJDJyaL
Wed May 18 13:59:14 +0000 2016 retweeted 3 favourited 2

Most Active Twitter Users:

Tweets
Pasi Jaakkola @PasiJaakkola

24

Gokhan Atil @gokhanatil
Oracle ACE Director, OCP, DBA, Founding member and Vice President of Turkey Oracle User Group, Blogger, Speaker, Enterprise Manager Cloud Control #EM12c #EM13c

20

Maris Elsins @MarisElsins
Oracle ACE, OCM,
Lead Database Consultant at Pythian

12

Osama Mustafa @OsamaOracle
Oracle Geek | Oracle ACE Director | Fusion middleware Specialist | Speaker | Cars Lover

10

Dear DBA Frank @fdernoncourt
Want to know more about the Oracle database? Follow me!

8

About the Lists:

  • The twitter data was fetched on May 22nd 2016. So the retweeted and favourited counts might have been changed.
  • Most favourited tweets are sorted based on favourited count (descending), retweeted count (descending) and tweet date.
  • Most retweeted tweets are sorted based on retweeted count (descending), favourited count (descending) and tweet date.
  • Most visible tweets are calculated based on the followers count of the original tweeter and retweeters.
  • Most active twitter users are calculated based on the total number of tweets and retweets.

Reading Data From Oracle Data Files (without Connecting Database)

$
0
0

I attended the BGOUG Spring Conference last week as a speaker. While in BGOUG Conference, I had a chance to listen Kamil Stawiarski’s session about hacking Oracle. Kamil spoke about potential security issues of high level privileges such as create any index and create directory. At the end of the session, he also showed a tool written by himself. The tool (I do not remember its name right now), reads a datafile, searches for object ID (of a table) and parse and list the rows of the table.

I am aware that Oracle Support has a similar utility, and as I heard it’s called “DUL” (Data UnLoader). DUL is used to export data from data files in case there’s no possibility to open the database because of corruption. So the concept wasn’t new to me, but seeing the tool in action, made me want to write my own tool which is capable of extracting data from data files.

Yesterday, when I’m back home from Bulgaria, I have started to develop my own tool in “C” language. Why did I pick “C”? Because I though it would be easier to handle data block structure using “C” pointers. After about 5 hours, I saw that I forgot “C language” at all. I had to check almost everything. Funny thing is, although I forgot “C”, I still managed to develop a working tool using “C”, thanks to Internet and stackoverflow.com 🙂

Anyway, the tool still needs lots of improvements and it’s far from useable state but it helped me to improve my knowledge about how Oracle stores data internally. So I’m very happy to start developing it, and I’ll keep developing it as a side project.

extractor

Here you can see the output of my tool (I call it “extractor”). On the left side, you see the data from HR.DEPARTMENTS table, and on the right side, my tool shows the same data, reading directly from the data file containing the table. For now, it can only handle varchars and small numerics. I have to say that “numeric values” are stored in a complex(!?) structure, I need to work on it to handle all numerics (including negative ones).

Will it be a open source project? Oh no! It’s totally spaghetti code! On the other hand, I’ll share my findings about how data is stored in data files in my future blog posts.

Addition: I got a question about how I figure out the structure of data files. I do not have access to any official documentation about internal structures of Oracle. I use “alter system dump datafile block” command to get dump of block, it contains valuable information, then I compare that information with real block content using a hex editor (in fact, I added DBF file to XCode project so I can open it as HEX). I also use DUMP() command, so I don’t need to insert all possible variations to disks, I can see the corresponding byte array of a value. So basically, the information I discover may be totally wrong but I do my best 🙂

BGOUG Spring Conference 2016 & Twitter Statistics

$
0
0

As I already mentioned in my previous blog post, I attended the BGOUG (Bulgarian Oracle User Group) Spring Conference last week. It was my second time to attend BGOUG conferences, and as usual, it was a great organization. I’m really impressed how Milena Gerova and BGOUG Conference Team can organize such a successful event.

presenting

At the conference, I gave two speeches, one was “Linux Essential Commands for DBAs”, and the other one was “Oracle Enterprise Manager 13c for DBAs”. The slides were very identical to the ones I used in my presentations I did on Finland, so I’ll not share them, but if you are a BGOUG member, you will be able to access them on BGOUG website. You can also access my previous presentations at slideshare.

dancers

When talking about this great conference, I have to mention the famous “BGOUG Appreciation Dinner”. It’s organized on Day 1. You can taste great food with local drinks, and try to perform Bulgarian folk dance! Unfortunately I got cold, so I hardly stay until I watched the show of the folk dancers, and then I went my room to get rest.

dinner

I had a chance to meet people I know from twitter and see old friends. Time is never enough to chat with all these folks, so hope to see them in future events.

It became a tradition for me to publish the twitter statistics of the conference. The official hashtag of the conference was #BGOUG, and here are the statistics:

#BGOUG Tweets

Number of tweets (including RTs):

243

Most Favourited Tweets:

Gokhan Atil @gokhanatil
I’m ready for my first session at #BGOUG Spring Conference. I’ll speak about #EM13c #Oracle https://t.co/NPMylAuxst
Fri Jun 03 07:23:08 +0000 2016 retweeted 1 favourited 20
Joze Senegacnik @joc1954
Having fun at best party at #BGOUG with @HeliFromFinland and @SQLMaria https://t.co/cKFW2y2dOs
Fri Jun 03 19:28:17 +0000 2016 retweeted 3 favourited 15
Gokhan Atil @gokhanatil
#BGOUG Speakers Dinner is about to start @horiaberca @ora600pl @pdevisser @joc1954 @SQLMaria @Mirela_RoOUG https://t.co/NGmpIx5Em8
Thu Jun 02 17:18:09 +0000 2016 retweeted 1 favourited 13
Joze Senegacnik @joc1954
@SQLMaria and @HeliFromFinland processing vectors in Oro dance #BGOUG https://t.co/l8i7RVmR6H
Fri Jun 03 19:33:54 +0000 2016 retweeted 3 favourited 12
Pasi Jaakkola @PasiJaakkola
@HeliFromFinland starting presentation at #BGOUG https://t.co/mx563hIhNE
Sat Jun 04 11:28:44 +0000 2016 retweeted 1 favourited 12

Most Retweeted Tweets:

Heli Helskyaho @HeliFromFinland
.@RoelH explaining the #UniversalTheme at #BGOUG #orclapex https://t.co/vvbPfL6KRd
Sat Jun 04 08:22:46 +0000 2016 retweeted 6 favourited 6
Zohar Elkayam @realmgic
Exploring Advanced SQL Techniques Using Analytic Functions #oracle #sql #oracle11gr2 https://t.co/F6JHng1BXO #bgoug /cc @bgoug @oracleace
Sat Jun 04 14:50:21 +0000 2016 retweeted 5 favourited 8
Zohar Elkayam @realmgic
When Bulgarian dance at the #bgoug dinner w/ @joc1954 @SQLMaria @HeliFromFinland @BigRedDW @gokhanatil @BGOUG https://t.co/B3zPXNwwJi
Fri Jun 03 19:07:36 +0000 2016 retweeted 5 favourited 7
Pasi Jaakkola @PasiJaakkola
Keynote #BGOUG starts soon. @Enkitec @AccentureTech https://t.co/Zraq8M65OF
Fri Jun 03 14:39:36 +0000 2016 retweeted 5 favourited 5
Zohar Elkayam @realmgic
Things Every #Oracle #DBA Needs to Know about the #Hadoop Ecosystem #bigdata https://t.co/0DZfeqW1wR #bgoug /cc @bgoug @oracleace
Sat Jun 04 14:52:34 +0000 2016 retweeted 5 favourited 5

Most Visible Tweets:

Zohar Elkayam @realmgic
Things Every #Oracle #DBA Needs to Know about the #Hadoop Ecosystem #bigdata https://t.co/0DZfeqW1wR #bgoug /cc @bgoug @oracleace
Sat Jun 04 14:52:34 +0000 2016 retweeted 5 favourited 5
Zohar Elkayam @realmgic
When Bulgarian dance at the #bgoug dinner w/ @joc1954 @SQLMaria @HeliFromFinland @BigRedDW @gokhanatil @BGOUG https://t.co/B3zPXNwwJi
Fri Jun 03 19:07:36 +0000 2016 retweeted 5 favourited 7
Heli Helskyaho @HeliFromFinland
.@RoelH explaining the #UniversalTheme at #BGOUG #orclapex https://t.co/vvbPfL6KRd
Sat Jun 04 08:22:46 +0000 2016 retweeted 6 favourited 6
Joze Senegacnik @joc1954
Having fun at best party at #BGOUG with @HeliFromFinland and @SQLMaria https://t.co/cKFW2y2dOs
Fri Jun 03 19:28:17 +0000 2016 retweeted 3 favourited 15
Heli Helskyaho @HeliFromFinland
I will be speaking about #BigData in Hall Borovets at 12.30 at #BGOUG . See you there?
Sat Jun 04 07:46:51 +0000 2016 retweeted 3 favourited 2

Most Active Twitter Users:

Tweets
Horia Berca @horiaberca
Core RDBMS and building PaaS knowledge on top of it. Enjoying my life @Oracle. Mens sana in corpore sano. #FatherOfAGirl. Views my own

16

Zohar Elkayam @realmgic
I’m an Oracle ACE Associate, Oracle DBA since 1998, senior consultant, blogger, and a public speaker. I’m also into Linux, MySQL, NoSQL, and Big Data

14

Gokhan Atil @gokhanatil
Oracle ACE Director, OCP, DBA, Founding member and Vice President of Turkey Oracle User Group, Blogger, Speaker, Enterprise Manager Cloud Control #EM12c #EM13c

12

Joze Senegacnik @joc1954
Oracle DBA,Developer,Performance troubleshooting specialist,Member of OakTable,Oracle ACE Director,Pilot (PPL/IR(SE)),singer,chef,baking bread, oven builder

11

Pasi Jaakkola @PasiJaakkola

10

About the Lists:

  • The statistics cover the period between June 2nd and June 5th, and the data was fetched on June 6th 2016. So the retweeted and favourited counts might have been changed.
  • Most favourited tweets are sorted based on favourited count (descending), retweeted count (descending) and tweet date.
  • Most retweeted tweets are sorted based on retweeted count (descending), favourited count (descending) and tweet date.
  • Most visible tweets are calculated based on the followers count of the original tweeter and retweeters.
  • Most active twitter users are calculated based on the total number of tweets and retweets.

How to Modify the Retention Time for Metric Data in EM13c

$
0
0

Some Enterprise Manager customers, complain about the high volume of data stored in OMS repository. Enterprise Manager agents collect metric data of targets they monitor, and upload these “raw” metric data to OMS. Depending on your metric collection periods and number of targets, the volume required to keep these data can be very huge. Enterprise Manager has already a mechanism to minimize the size of the Management Repository. Once a day, the previous day’s metric data (which is stored in em_metric_values) is aggregated into a one-hour and a one-day table (em_metric_values_hourly, em_metric_values_daily). According to the retention settings, both raw metric data and aggregated data are purged.

The default retention time for raw metric data is set to 7 days. The default retention time for hourly aggregate metric data is set to 31 days, and the default retention time for daily aggregate data is set 12 months. You can check the current retention using get_retention function in the gc_interval_partition_mgr package:

-- returns days
select gc_interval_partition_mgr.get_retention('SYSMAN', 'EM_METRIC_VALUES')
from dual;

-- returns days
select gc_interval_partition_mgr.get_retention('SYSMAN', 'EM_METRIC_VALUES_HOURLY')
from dual;

-- returns months
select gc_interval_partition_mgr.get_retention('SYSMAN', 'EM_METRIC_VALUES_DAILY')
from dual;

You can also directly query the em_int_partitioned_tables but the table names can be slightly different depending on the version of Enterprise Manager. For example, for EM13c, you need to add “_E” to the table names.

SELECT table_name,
  partitions_retained
FROM em_int_partitioned_tables
WHERE table_name IN ('EM_METRIC_VALUES_E', 
'EM_METRIC_VALUES_HOURLY_E', 
'EM_METRIC_VALUES_DAILY_E');

If you have problem with the size of the Management Repository. you can modify the retention for metric tables using set_retention procedure of the gc_interval_partition_mgr package:

begin

-- sets to 5 days
execute gc_interval_partition_mgr.set_retention('SYSMAN',
'EM_METRIC_VALUES', 5 );

-- sets to 21 days
execute gc_interval_partition_mgr.set_retention('SYSMAN',
'EM_METRIC_VALUES_HOURLY', 21 );

-- sets to 6 months!
execute gc_interval_partition_mgr.set_retention('SYSMAN',
'EM_METRIC_VALUES_DAILY', 6 );

end;

Although I share this information, I recommend you to not to modify the retention time of raw metric data. Maybe you can modify retention time for hourly and daily aggregated data. The best way to deal with high volume of metric data, is to modify monitoring templates and disable unnecessary metric collections and reduce “the collection frequency” for unimportant metrics.

Installing EM13c Agents on Microsoft Windows Target Hosts

$
0
0

If your EM13c OMS installed on Unix-based OS, and want to deploy agents to Windows servers, you need to install Cygwin to target windows server to use regular deployment process. You can also use PsExec utility but in this case, you need a Microsoft Windows staging server. Installing Cygwin to enable SSH on Windows may sound like an unnecessarily complex solution at first, but Cygwin is easy to install and it comes with lots of useful utilities. So It’s worth installing. In this blog, I’ll show how to deploy EM agent on Microsoft Windows target. I’ll download and install Cygwin to the target server, enable SSH service and create user for SSH. After these steps, I’ll use Enterprise Manager Add Host page to deploy Enterprise Manager agent to the target Windows server like it’s just another Linux host.

First login to the target Windows server with administrator user and download Cygwin from https://www.cygwin.com/install.html. Depending on your CPU architecture (is there any one still using 32bit?) you need to download setup-x86.exe or setup-x86_64.exe from the installation page. These are very small executables, and they will download the required packages from the internet.

cygwin2

When you run the setup-x86_64.exe, you see the welcome screen. You click NEXT to choose a download source. If your target server has no internet access, you can install packages from a local directory. That means, you can use the packages downloaded by another Cygwin installation on another server. All you need is to use a shared location or copy files from other server which is already Cygwin installed. Click NEXT to continue.

cygwin3

In this screen you set the installation path. Be careful that, Cygwin 64bit’s default installation path is “C:\Cygwin64”, but in “ssPaths_msplats.properties” file of Oracle Enterprise Manager, Cygwin path is defined as “C:\Cygwin” so So if you use a different path than “C:\Cygwin”, you need to modify all path variables in the file ($OMS_HOME\oui\prov\resources\ssPaths_msplats.properties). At first, I left it as it is, then I had to modify the “ssPaths_msplats.properties” file:

# Copyright (c) 2005, 2013, Oracle and/or its affiliates. All rights reserved.

IDENTITY=identity
IDENTITY_PUB=identity.pub
FALSE=C:/cygwin64/bin/false.exe
#SSH_PATH=C:\\Program Files\\OpenSSH\\bin\\ssh.exe
#SCP_PATH=C:\\Program Files\\OpenSSH\\bin\\scp.exe
SSH_PATH=C:/cygwin64/bin/ssh.exe
SCP_PATH=C:/cygwin64/bin/scp.exe
PING_ARGS=-n 5 -w
PING_PATH=%SYSTEMROOT%\\system32\\ping.exe
LS_PATH=C:/cygwin64/bin/ls.exe
LS_ARGS=-A
MKDIR_PATH=C:/cygwin64/bin/mkdir.exe
MKDIR_ARGS=-p
ZIP_PATH=C:/cygwin64/bin/zip.exe
UNZIP_PATH=C:/cygwin64/bin/unzip.exe
...

After setting the correct path, click NEXT button. It will ask you where it should download the packages, how your server connects to internet, which download site should be used. Usually it’s okay to accept the defaults until you see “select packages” screen.

cygwin7

All you need is to select “openssh and openssl” packages. So type “opens” into the search box, expand the “Net” category, and select the openssh and openssl packages. Cygwin installer will detect the additional required packages, just accept them and click NEXT until installation start, then wait for downloads and installations of packages are done.

After the installation is done, right-click on Cygwin64 Terminal icon on the desktop and choose “run as administrator”. So the terminal will start with admin rights.

Run ssh-host-config and give the following answers to the questions:

Query: Should StrictModes be used? (yes/no) yes
Query: Should privilege separation be used? (yes/no) yes
Query: new local account ‘sshd’? (yes/no) yes
Query: Do you want to install sshd as a service? yes
Query: Enter the value of CYGWIN for the daemon: []
Query: Do you want to use a different name? (yes/no) no
Query: Create new privileged user account ‘SQL01\cyg_server’ (Cygwin name: ‘cyg_server’)? (yes/no) yes
Query: Please enter the password: password
Query: Reenter: password

It should say “Host configuration finished” at the end. In my installation, it couldn’t find “cyg_server” for SSHD although it created, so it used “SYSTEM” user. That’s fine. Now open “C:\cygwin\etc\sshd_config” file (you can use vi or any other editor). Add the following line at the end of the time:

KexAlgorithms diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1

Then start SSH service:

$ net start sshd
The CYGWIN sshd service is starting.
The CYGWIN sshd service was started successfully.

For now, you do not have any users in Cygwin environment. I’ll add my current (local) user to the Cygwin environment so we’ll be able to use it when deploying agent.

/bin/mkpasswd -l -u dbadmin >> /etc/passwd

$ cat /etc/passwd
SQL01+dbadmin:*:197613:197121:U-SQL01\dbadmin,XXXXXXXXXX:/home/dbadmin:/bin/bash

As you can see I checked if there’s any line added to /etc/passwd. If you see your username there, you should be able to connect the target Windows server using SSH.

addtargets

Now we need to login to EM13c console, go to “Add Targets Manually” page, click “Install Agent on Host”.

addhost1

Enter host name and select the platform, click NEXT to go to “installation details” screen. Of course, you should have already installed the agent for Windows platform to be able to choose it on “the platform select box”.

addhost2

Enter the path in Windows format, for example: “C:\oracle\emagent”. Enter the username you added to “/etc/passwd” on the target Windows server. Click “NEXT” and deploy the agent. That’s all. Now we can deploy Microsoft SQL Server plugin to this agent and monitor our Microsoft SQL Servers, but I’ll tell it on another blog post 🙂

Oracle Enterprise Manager Plugin for PostgreSQL

$
0
0

Oracle Enterprise Manager supports 3rd party plugins which can help you to extend its monitoring and manageability capabilities. There are already developers and companies specialized in plugin development. Blue Medora is one of them (and probably the most famous one). In this blog post, I’ll show you how we can use Blue Medora’s EM PostgreSQL plugin, which is part of their True Visibility Suite for Oracle. I’ll download and deploy the plugin on the OMS (management servers), then deploy the plugin on the agent(s) and at the end, I’ll add a PostgreSQL target to our Enterprise Manager system.

The PosgteSQL plugin is not available on Self-Update, so we need to download and register it manually to our Enterprise Manager. To download the trial version of the plugin, I go to their website, click “download trial”, and fill a form to request trial version. Blue Medora will see our form and send us an email with download link of the plugin. After I download the zip file to my OMS server (ie into the temp folder), I issue the following commands:

cd /tmp
unzip bm.em.xpgs_12.1.0.2.0_2.0.3.zip -d postgre_plugin
emcli login -user=SYSMAN
emcli import_update -omslocal -file=/tmp/postgre_plugin/bm.em.xpgs_12.1.0.2.0_2.0.3.opar

postgres1

Then I login to Enterprise Manager console, go to “Extensibility > Self Update” page. Click to the “plugins”, find PostgreSQL plugin in the list, select the plugin and “apply it” (it’s in the actions menu on the left side).

deploytoagent

After I applied the plugin, it’s ready to deploy on OMS (we need to deploy it on OMS before we can deploy it on the agents). I go to “Extensibility > Plugins” page, find the PostgreSQL pluin in the database plugins, click “deploy on” and select “management servers”. It’s a very straightforward process. EM will check the plugin and its prerequisites and if everything is OK, I just click NEXT to reach the last page and click DEPLOY button to start deployment of plugin on the OMS.

pgdepagent4

After I deployed the plugin on OMS, I can start deploying the plugin on the agents. Blue Medora’s PostgreSQL plugin can monitor PostgreSQL databases remotely but I recommend you to do local monitoring. So I’ll deploy the plugin on the agent running on the same host with the PostgreSQL database.

To deploy on the agents, I click “deploy on” menu and then select “management agent”. It will show us the Plugin version, ask the target agents (hosts), do some prerequisite checks on the agents and then show us the review page. I click DEPLOY after reviewing it. We can monitor the status of the deployment operation in the Deployment Activities page.

addpostgretarget1

After the plugin is deployed on the agent, now we need to use “Add Target Declaratively” to add PostgreSQL database targets. I go to “Add Targets Manually” page, click on “Add Target Declaratively”. Select the host (the on you already deployed the plugin), select the target type as “PostgreSQL Database”, click “Add” button.

addpostgretarget2

The PostgreSQL plugin needs only a few information about the target database. We should give an unique name for the target, hostname of the database (localhost should be fine for local monitoring but do not forget to modify pg_hba.conf file to give permission to access the plugin), login credentials, and primary database. Primary database is the first database the plugin will show on the overview page. If you’re not sure about picking which database, you may enter “postgres”. Don’t forget to click “test connection” to see if the plugin can connect to the server. If it works, you can click OK to add the target.

postgreshome

This is the target home page of PostgreSQL database. It provides overall health of the databases. Some important indicators related with target status (such as indexes, triggers, number of connections, prepared transactions, autovacuum freeze), latest incidents, currently open sessions, replication status and background writer status are reported in this page. You may see the page almost empty right after you added the target. Please be patient and give some time to the plugin to populate all data it requires.

databaseinfo

There are four tabs for each database. Let’s check each of them to see which information the plugin provides to us. In the “database” tab we will see a pie chart at the top left, showing the ratio between the tables and the indexes on the database. On the right side, there’s a chart showing statistics about the database such as database size, number of transactions, blocks reads vs hits, block read time vs write time, row access counts, deadlocks and conflicts. We can pick one of these statistics from the select box and the chart will be updated. At the bottom side, there’s a table showing current connections to the selected database.

tableinfo

In the “tables” tab, we see a pie chart showing the biggest tables of the database. On the right side, there’s a chart showing scan counts, rows fetched by scan type (very important statistics!), rows modified, live and dead rows (tuples), table maintenance, heap blocks, index blocks and toast blocks. On the bottom, there’s a grid report listing all the tables. We can sort these tables according to their names, schemas, types, number of rows, disk space, and accessed rows per hour. When we click on a table name, the statistic chart will be updated. There are really useful statistics about the tables so can find most actively used tables and see their workload characteristics.

indices

In the “indices” tab, we can see the biggest indexes, most actively used indexes, how many rows (disk blocks) are fetched for each index.

queriesinfo

The last tab is the “queries” tab. Be sure to enable pg_stat_statements extension on your PostgreSQL server to be able to use this tab. In the “queries” tab, we can see most executed queries and valuable statistics for each of these queries such as average execution time, number of rows returned/affected, call count, block read and write time, shared blocks.

The PostgreSQL plugin comes with more than 100 metrics. Using The Enterprise Manager target menu, we can check these metrics, set thresholds, manage incidents. We can also manage blackouts, use Information Publisher Reports to generate reports about PostgreSQL targets.

I found Blue Medora’s PostgreSQL plugin is really useful. It helps you to easily monitor your PostgreSQL server, and notice issues about your databases which you do not normally see (unless you’re an expert PostgreSQL DBA). So I recommend it to all Enteprise Manager admins managing PostgreSQL databases.

Book Review: Oracle Database Cloud Cookbook with Oracle Enterprise Manager Cloud Control 13c

$
0
0

13cCoverLatest Last week, I had a chance to read the latest book of my friend Porus Homi Havewala. If you’re interested in Oracle Enterprise Manager, you already knew him or at least read one of his articles/blog posts. He’s a Subject Matter Expert (SME) on Oracle Enterprise Manager Technology, Oracle Certified Master (OCM) and Oracle ACE Director. Porus created one of the world’s first blogs dedicated to Enterprise Manager (http://enterprise-manager.blogspot.com), and he is the author of the books, “Oracle Enterprise Manager Grid Control (published by Rampart)” and “Oracle Enterprise Manager Cloud Control 12c: Managing Data Centre Chaos (published by PACKT)”.

His latest book also focus about Oracle Enterprise Manager and titled as “Oracle Database Cloud Cookbook with Oracle Enterprise Manager Cloud Control 13c“. It’s a cookbook so the readers will get fast recipes to build your Hybrid Database Cloud using EM13c.

The book has 382 pages and there are 8 chapters:

1) Consolidation Planning for the Cloud: This chapter focuses on two tools offered by the Enterprise Manager: The Host Consolidation Planner and the Database Consolidation Workbench. In this chapter, you’ll see sample usage of both tools.

2) Database as a Service: Oracle Enterprise Manager Cloud Control offers many capabilities for Oracle’s private database cloud services. In this chapter, Porus explains how to create the required PaaS infrastructure, database pool, self-service portal and configure chargeback. All topics are explained with samples and screenshots. Specially the PaaS infrastructure is very important because it provides the base platform for all software services.

3) Schema as a Service: The foundation of this service is a schema pool of pre-created Oracle Database instances. In this chapter, we can see how to set up schema pools, quotas, data profiles, service templates, request settings, and, optionally, chargeback plans. Schema as a Service can be very useful for mid-sized companies.

4) Pluggable Database as a Service: Multi-tenant architecture and Pluggable Databases are one of the top features of Oracle 12c. Enterprise Manager also supports Pluggable Database as a Service. In this chapter, Porus shows us how to set up Pluggable Database Pool, set quotas and service templates and configure chargeback for PDBaaS.

5) Hybrid Database Cloud: Sooner or later, we will start to use Cloud Databases. As you may know, Oracle Enterprise Manager 13c can monitor and manage both Cloud and on-premise databases. This chapters covers the topics such as installing Entperirse Manager agents, discovering Cloud databases (to add our Enterprise Manager system), cloning databases between Cloud and on-premise.

6) Using the Cloud REST API: If our company doesn’t want to use the Enterprise Manager Self Service Portal, we can create our own self service portal using the Cloud REST API of Enterprise Manager. All samples are given as direct REST API calls using Postman REST client, so we do not need to know a specific programming language to understand the samples.

7) Managing Database Backups: Oracle Enterprise Manager enables centralized control of database backups. This chapter covers how we can use Oracle Enterprise Manager to set up and schedule RMAN backups (both incremental and full) for all the databases in a Cloud environment, and howe we can generate backup reports.

8) Managing Standby Databases: The web interface provided by Oracle Enterprise Manager for creating and managing Data Guard configurations is extremely easy to use. This chapter demonstrates how we can set up, manage, and monitor standby databases for any primary database using Oracle Enterprise Manager Cloud Control 13c.

I can say that I really liked the book. It’s really easy to read and all topics are explained by detailed samples. Porus Havewala has a deep knowledge and experience about the subject so his instructions are very clear. It’s like following a workshop about building Database as a Services architecture on Cloud.

How to Import Data (to Oracle RDBMS) from XML

$
0
0

I wrote a blog post about how to import data from a XML file to Oracle about 7 years ago. I demonstrated how we can use DBMS_XMLSTORE package to parse XML data. Yesterday, I replied a question from a blog reader about that post. He had problem with running my sample script, I did troubleshoot and find out the problem with his script, and while testing the sample script, I noticed that I can write much better script.

This is my sample XML (employees.xml):

<EMPLOYEES>
<EMPLOYEE>
<NO>1</NO>
<NAME>Gokhan Atil</NAME>
</EMPLOYEE>
<EMPLOYEE>
<NO>2</NO>
<NAME>Ned Flanders</NAME>
</EMPLOYEE>
<EMPLOYEE>
<NO>3</NO>
<NAME>Agnes Skinner</NAME>
</EMPLOYEE>
<EMPLOYEE>
<NO>4</NO>
<NAME>Milhouse Van Houten</NAME>
</EMPLOYEE>
</EMPLOYEES>

I created a table to load the data and a directory link so we can access the file:

CREATE TABLE empfromxml ( no NUMBER, name VARCHAR2(50 ));
CREATE DIRECTORY xmltest AS '/oracle/xmltest';

Instead of using DBMS_XMLSTORE, I decided to use XMLType functions ExtractValue and XMLSequence. So I modified my old sample script and wrote this one:

DECLARE
xml_file BFILE;
xml_data CLOB;
BEGIN
xml_file := BFILENAME ('XMLTEST', 'employees.xml');
DBMS_LOB.createtemporary (xml_data, TRUE, DBMS_LOB.SESSION);
DBMS_LOB.fileopen (xml_file, DBMS_LOB.file_readonly);
DBMS_LOB.loadfromfile (xml_data, xml_file, DBMS_LOB.getlength(xml_file));
DBMS_LOB.fileclose (xml_file);

INSERT INTO EMPFROMXML (no, name)
    SELECT ExtractValue(Value(x),'//NO') as no,
           ExtractValue(Value(x),'//NAME') as name
    FROM   TABLE(XMLSequence(Extract(XMLType(xml_data),'/EMPLOYEES/EMPLOYEE'))) x;

DBMS_OUTPUT.PUT_LINE( SQL%ROWCOUNT || ' rows inserted.' );
DBMS_LOB.freetemporary (xml_data);
COMMIT;
END;
/

You can compare it with my previous one. As you can see, I removed DBMS_XMLSTORE functions and write a simple SQL to fetch data from a CLOB which contains XML data. The main process is just an INSERT query but there are still some ugly stuff because I need to deal with reading file to the CLOB variable.

Then I realized that I could just remove all the CLOB stuff. In fact, a simple SQL can handle all the task:

INSERT INTO EMPFROMXML (no, name)
    SELECT ExtractValue(Value(x),'//NO') as no,
           ExtractValue(Value(x),'//NAME') as name
    FROM   TABLE(XMLSequence(Extract(XMLType(bfilename('XMLTEST', 'employees.xml'), 
    nls_charset_id('UTF8') ),'/EMPLOYEES/EMPLOYEE'))) x;

This code will read from employees.xml file in XMLTEST directory, parse the data (the rowtag is EMPLOYEE, the fields are NO and NAME) and insert it into the EMPFROMXML table.

xmltype

Very clean and simple!

EM13c: Performance Tuning Access to Read Only Administrators

$
0
0

About six months ago, when I was in the OUGN Spring Conference, someone asked me how they can let developers access to performance tuning pages on Enterprise Manager Cloud Control 13c. He said that they granted full administrator privileges to the developers so they could access the performance page but it’s not acceptable. He also mentioned that they didn’t need to grant full administration privilege to the developers’ EM users while they were using EM12c.

Unfortunately I forgot to examine the situation until I saw a similar question on OTN (Oracle Community) forum on Friday. The forum reader has exactly the same problem. The database user was already granted required privileges but the EM administrator see the menu items related with “performance tools” are grayed out.

Before going into further details, let me explain the difference between the EM administrator (user) and the database user. You login to the EM13c web console using an EM administrator. EM administrators are defined in the EM environment. When you want to manage (or monitor performance page) of a database target through Enterprise Manager, it will ask you to enter the credentials of a database user. So although you need only one EM administrator account, you need to have one database user for each database target. The named credentials feature can be used to store and share these database credentials in the EM repository.

Okay, so we want our non-administrative/read-only EM administrators (i.e. developers) access performance pages and tools of a database and we want to do it according to the principle of least privilege.

First, let’s create a database user on the target database. Following SQL script can create a database user (DBPERFUSER) who can access “Database Performance Page”, “AWR/ADDM” and Enterprise Manager advisors (such as SQL Access advisor, SQL Tuning advisor etc):

create user DBPERFUSER identified by mysuperscreetpassword;
grant create session to DBPERFUSER;
grant select any dictionary to DBPERFUSER;
grant execute on dbms_workload_repository to DBPERFUSER;
grant oem_advisor to DBPERFUSER;

userproperties

After I created a database user on the target database, I login to EM13c web console, go to “security > administrator” page and create an EM administrator (user).

userroles

I grant only the default roles (EM_USER and PUBLIC). For now, this user can login to Enterprise Manager console but they can not monitor or manage any target.

addtarget

I need to add the target database for granting target privileges. I clicked the pen icon to go the target privileges page.

targetpriv

In the target privileges page, I granted “Manage Database Performance Privilege Group” to the EM administrator for the target database.

review

I review and create the EM administrator. The newly created administrator will be able to access the performance pages and tools for the specified database target if they login with the database user (DBPERFUSER).

Speaking at OOW 2016: Write powerful scripts with EMCLI

$
0
0

This will be my first time to attend Oracle Open World, and I’ll speak about Enterprise Manager Command-Line Interface (EMCLI) Scripting” on User Group Sunday. I already presented it on several conferences, heard people’s feedback and modified the content to improve my presentation. Now I will focus more on practical information, share sample scripts that can be used in real world scenarios. Hope to see you there!

Session ID: UGF3543
Session Title: Oracle Enterprise Manager Cloud Control: Write Powerful Scripts with EMCLI
Room: Moscone West – 3022
Date and Time: 09/18/16, 11:45:00 AM – 12:30:00 PM

Write powerful scripts with EMCLI

Don’t forget to add my session to your schedule to reserve your seat! 🙂


EM13c: Unique Database Service Names on DBaaS

$
0
0

When creating Database through Oracle Enterprise Manager Self Service Portal, we need to enter a “unique” value as database service name. After I set up DBaaS on EM13c, I made several tests until I fix the provision problems (based on my own faults). On each test, I cleaned up failed request, entered a unique service name and submitted a new request. I thought there’s no validation about the service name but then I got an error message complaning “non-unique service name”.

remove_reserved_artifact

As we can see, I tried to enter “murtaza” and it says “murtaza” is not available. I knew that I have already cleaned up failed requests, and there is no service named “murtaza”. Anyway I wanted to be sure and decided to check the existing database services. I noticed that there’s no page showing the existing service names! So I had to enter each database service and check the service name. I had only 3 database services, so it didn’t take so much time.

connection string

The connection string (which we’ll share with the request owner), contains the service name. I couldn’t find any service named “murtaza”. It must be used for a failed request. As I said, I have already cleared and deleted all failed requests, but somehow the service name was stored in the repository. Good thing is, EMCLI verb “dbaas_artifacts” can show and remove these service names which belongs to failed requests:

# login to OMS as gokhan
./emcli login -username=gokhan

# check the names
./emcli dbaas_artifacts -list_reserved_artifacts 

List of reserved artifacts:

Reserved Name: MURTAZA, Type: SERVICE_NAME, Request Id: 21

# remove the name
./emcli dbaas_artifacts -remove_reserved_artifact -type="SERVICE_NAME" -value="MURTAZA"

Removing the artifact - MURTAZA (SERVICE_NAME) ...
Successfully removed the artifact - MURTAZA (SERVICE_NAME).

The “list_reserved_artifacts” parameter lists the reserved names. If there’s on going provision process, its name can also be seen here. If we want to remove any reserved service name, we should use “remove_reserved_artifact” parameter. It seems sometimes, cleaning up and deleting a failed request using the web interface, might not clean all the traces. Also the service names of “rejected” requests are not cleared even we delete the request from the web interface. So I recommended to get familiar with emcli verbs to clean up the unused service names.

Because I removed the reserved service name, I could continue to create a new database service with the same name. To test the validation on service names, I tried to enter an existing service name. Guess what happened? The validation mechanism which checks “reserved service names”, does not check the existing service names!

So I have created several database services with same name. DBaaS created separate databases with unique SIDs for them, but they have same service name. The problem occurs if some of these databases are created on the same host in the database pool, because they will share same listener and register themself as same service name:

Listener Parameter File   /u01/oracle/grid/12.1.0.2/grid/network/admin/listener.ora
Listener Log File         /u01/oracle/diag/tnslsnr/xxxxxx/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=xxxxxx.com)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM", status READY, has 1 handler(s) for this service...
Service "TESTDBSTB" has 1 instance(s).
  Instance "TESTDB", status READY, has 1 handler(s) for this service...
Service "TESTDBSTB_DGB" has 1 instance(s).
  Instance "TESTDB", status READY, has 1 handler(s) for this service...
Service "db000000" has 1 instance(s).
  Instance "db000000", status READY, has 1 handler(s) for this service...
Service "db000002" has 1 instance(s).
  Instance "db000002", status READY, has 1 handler(s) for this service...
Service "db000004" has 1 instance(s).
  Instance "db000004", status READY, has 1 handler(s) for this service...
Service "ilayda" has 2 instance(s).
  Instance "db000002", status READY, has 1 handler(s) for this service...
  Instance "db000004", status READY, has 1 handler(s) for this service...
Service "testere" has 1 instance(s).
  Instance "db000000", status READY, has 1 handler(s) for this service...
The command completed successfully

Their connection string (which will be shared with the request owner) will be also same, and it can cause serious problems. Oracle surely need to fix the validation mechanism for the service names. Until that time, we should need to find a way to see the existing service names, so as a Cloud Administrator, we can approve the request or ask them to find a new service name.

COLUMN target_name FORMAT A15
COLUMN service_name FORMAT A15
COLUMN HOST_NAME FORMAT A30

select p.target_name, p.property_value service_name, t.HOST_NAME 
from SSA_DBAAS$REQUEST_PROPERTIES p,
MGMT$TARGET t where 
t.TARGET_GUID = p.TARGET_GUID and
p.property_name = 'DB_SERVICE_NAME'
order by service_name, host_name, target_name;

TARGET_NAME     SERVICE_NAME    HOST_NAME
--------------- --------------- ------------------------------
db000001        deep            yyyyyy.com
db000003        ilayda          yyyyyy.com
db000002        ilayda          xxxxxx.com
db000004        ilayda          xxxxxx.com
db000000        testere         xxxxxx.com

I connected to the repository database and find a view (SSA_DBAAS$REQUEST_PROPERTIES) which holds the service names used with DBaaS. I joined it with MGMT$TARGET to add host name to the result. The above query also can be useful when we need to remove a database service (on DBaaS) and all we know is its service name.

Oracle Enterprise Manager Cloud Control: Write Powerful Scripts with EMCLI

$
0
0

Last week, I attended the Oracle Open World and did a presentation about writing scripts with EMCLI. If you’re not familiar with EMCLI, it’s the command line interface for Oracle Enterprise Manager Cloud Control. Here’s my presentation:

Although EMCLI is a very specific topic which appeals only to advanced users, a lot of people attended my session. I would like to thank Ray Smith (IOUG Director of Education) for his support. He did his best to inform people about my session.

If you attended my session or just checked the presentation slides, and have any questions about EMCLI scripting, please do not hesitate to ask me.

OTN Appreciation Day: Oracle Wait Interface

$
0
0

Thanks to Tim Hall, lots of people from Oracle community, shared their favourite Oracle feature as blog posts. I’ve decided to dedicate my OTN Appreciation Day blog post to Oracle Wait Interface. I know that the topic was not limited to Oracle database, and I could share something about Oracle Enterprise Manager (my main focus area), but Oracle database is the entry point for me and it’s the reason why I started to use other Oracle products (including Oracle Enterprise Manager).

Most of the time, people ask me why I prefer to work with Oracle database. It has great features like having reliable infrastructure, flashback, partitioning, in-memory column store etc. It comes with a great backup and recovery tool (RMAN) and it supports PL/SQL (great way to deal with data). All of these features (and the ones I didn’t count here) helps Oracle database to be the best relational database management system, but when I think about Oracle, the performance monitoring features are my favourite, and Oracle Wait Interface is the foundation stone of those features.

Each time a process waits for something, Oracle collects statistics about the wait. Oracle Wait Interface provides statistics for these waits, and these statistics can be accessed trough a collection of dynamic performance views or an extended SQL trace file. This information is invaluable to monitor sessions and guides you to find the root cause of many performance problems.

For more information about Oracle Wait Interface, you may read: “Oracle Wait Interface: A Practical Guide to Performance Diagnostics & Tuning” book.

EM13c: Is It Possible to Delete a Job Run with EMCLI?

$
0
0

Enterprise Manager Cloud Control provides a centralized job scheduling system and it’s possible to control it through both web interface and EMCLI. On the other hand, it seems web interface provides more control over the past jobs, for example it’s not possible to delete a job run with EMCLI, while it’s possible to do it on web interface:

deleterun

EMCLI verb “get_jobs” reports all runs of the jobs, and “delete job” can delete job and its all runs but it’s not possible to delete a specific run of a job. For example, if you want to delete hundreds of “failed” (or “skipped”) runs of a job, EMCLI won’t help you and doing it through the web interface will be very time consuming.

So what is the solution? We need to make our hands dirty and call EM_JOB_OPS.delete_job_run procedure directly on the repository database.

Here’s a simple PL/SQL script to clean up failed jobs of a user:

BEGIN
  FOR c IN
  (SELECT scheduled_time
  FROM MGMT$JOB_EXECUTION_HISTORY
  WHERE job_owner=:1
  AND job_id     =:2
  AND status     ='Failed')
  LOOP
    EM_JOB_OPS.delete_job_run( :2, c.scheduled_time );
  END LOOP;
  COMMIT;
END;

Login to the Enterprise Manager repository database as SYSMAN, enter “job_owner” and “job_id” as bind variables. It will delete all failed runs of the job. You can query MGMT$JOB_EXECUTION_HISTORY view or use “emcli get_jobs” to learn the job owner and job ID information.

EM13c: Get List of URLs Monitored by Beacons

$
0
0

When you need to monitor web services, you can use EM beacons. A beacon is a target that allows the EM Agent to remotely monitor services. You can add beacons to any EM agent using “Add Non-Host Targets by Specifying Target Monitoring Properties” option in the Add Targets Manually page. After adding a beacon, you can configure it to fetch a web page and test the status of the service according to the result of the web transaction.

One of my blog reader said that they have a list of URLs and they need to know which beacon server is polling each URL by querying the URL. We need to query four views to get this information: mgmt_targets, mgmt_bcn_target, mgmt_bcn_step_defn, mgmt_bcn_step_props.

llistofurls

You can see the relation between these views in the diagram, and here’s the list of URLs monitored by all beacons:

SELECT t.HOST_NAME,
  t.TARGET_NAME,
  bsp.STRING_VALUE,
  bsp.NAME
FROM MGMT_TARGETS t,
  MGMT_BCN_TARGET bt,
  MGMT_BCN_STEP_DEFN bsd,
  MGMT_BCN_STEP_PROPS bsp
WHERE bt.BEACON_TARGET_GUID = t.TARGET_GUID
AND bsp.STEP_GUID = bsd.STEP_GUID
AND bsd.TARGET_GUID = bt.TARGET_GUID
AND bsp.NAME = 'url';

Viewing all 163 articles
Browse latest View live