Recovering Informix

 Recovering Informix is much easier than recovering other databases. One reason is that the commands to actually perform the recovery are simple---there is only one argument for ontape and only two arguments for onbar. This section covers just about any recovery scenario that you might find yourself in, yet it's only 20 main steps.

Another reason that Informix recoveries are simple is that the sysmaster database, physical log, and logical log are considered critical. In order to recover one of them, you have to recover all of them. In Oracle, for instance, there are four or five different recovery tactics that you can take, depending on which one of these objects is damaged. With Informix, there is only one recovery tactic.

The Informix recovery procedure does not assume that you know why your database went down. It does assume that you have been making backups via ontape or onbar. It also assumes that if you are using ontape, you know which tapes or files contain the latest level 0 and/or level 1 backups, as well as the location of all logical log backups since your last physical backup. If you are using onbar, this procedure assumes you know how to use your storage manager well enough that you know how to respond to its prompts for any media that may not be in an autochanger. In short, all media management is up to you.

The examples below use ontape. The example hostname is curtis, and the instance name is crash. The archives are being sent to disk files named /informix/logical/crash/crash.level.level.Z, and continuous backups are being sent to a disk file named /informix/logical/crash.log.

You should start with Step 1, "Does oninit work?"

Step 1: Does oninit Work?

The obvious first step in determining if an instance is in need of recovery is to try and start the instance. Do this by issuing the oninit command with no options. If it works, it just returns the prompt to you. You could also see one of two errors.

curtis $ oninit

WARNING: Cannot access configuration file $INFORMIXDIR/etc/$ONCONFIG.

This error is pretty obvious. If you see it, you are missing your configuration file. If you see the error above, proceed to Step 2. oninit: Cannot open chunk '/informix/rootdbs.dbf'. errno = 2

oninit: Cannot open chunk '/informix/rootdbs_mirror.dbf'. errno = 2

oninit: Fatal error in shared memory initialization

If any of your critical dbspaces has damaged or missing chunks, you may see an error like the one above. If all critical dbspaces are mirrored, and only one-half of the mirror is damaged, you do not see this error. It also does not appear if a non-critical dbspace is damaged. This error appears only if all chunks in a critical dbspace are damaged. If you see an error like the one above, proceed to Step 4. If you don't see either error, proceed to Step 7. Step 2: Is the onconfig File Missing?

The oninit utility uses the onconfig file to determine the basic information needed to start the instance. This includes, but is not limited to, the following instance-specific information:

If the onconfig file is missing or corrupted, proceed to Step 3. If this is not the reason the instance would not start, proceed to Step 4. Step 3: Restore or Recreate the onconfig File

If you are running onbar, it can automatically recreate the onconfig file. However, if this file is the only one damaged, there's no need to do a full restore just to restore this file. Restoring or recreating it is easy enough.

If you are running infback.sh, it makes a backup copy of the onconfig file before it changes it. DBAs and other scripts often do the same. Look first to see if you have such a backup copy. If not, try to restore the file from the nightly filesystem backups. If you cannot find a backup copy, and cannot restore one from backup, you will need to recreate it. If any of the following objects is available, it will be easy:

To recreate the onconfig file from the root dbspace chunk or an ontape archive on disk, run the following command, where filename is the name of the chunk or archive on disk: $ strings filename|grep '^[A-Z][A-Z_]*' \

>$INFORMIX/etc/$ONCONFIG

If you have only an archive available on tape, the command is similar: $ dd if=$TAPEDEV bs=$TAPEBLK \

| strings |grep '^[A-Z][A-Z_]*' \

> $INFORMIX/etc/$ONCONFIG
This creates an editable text file that contains all the parameters and their current values. The grep command does not completely remove extraneous lines, so you should edit it and remove any lines that do not look like the following: PARAMETER value If you do not have one of these objects available, you need to manually recreate the file. Copy the onconfig.std file in $INFORMIXDIR/etc to the name of the onconfig file. The values that you must change are ROOTNAME, ROOTPATH, DBSERVERNAME, SERVERNUM, and SHMBASE. These values will allow you to restore the instance.

Step 4: Is there an Inaccessible or a Critical Chunk?

If an Informix instance will not start, the most common cause is a missing or corrupt critical chunk. (If a non-critical chunk is damaged the instance starts and records the problem to the online log file.) The error that you receive may look something like the following:

oninit: Cannot open chunk '/informix/rootdbs.dbf'. errno = 2 According to errno.h, an "Error 2" means "No such file or directory." This means that the chunk, or the symbolic link that Informix uses to refer to that chunk, is missing. Another common error is 13, which means "Permission denied." This means that someone other than Informix owns the device. Any error other than those usually means that the physical file is all right, but the data within it is corrupted. If the file is missing or its permissions are wrong, proceed to Step 5. If not, proceed to Step 6. Step 5: Repair or Replace the Missing Chunk

This step is necessary only if the physical file is somehow damaged. If it was a filesystem file, it might be deleted or its permissions changed. If it was a raw device, the disk drive could be damaged or missing, or its permissions could be wrong. Another problem could be that you are using a symbolic link to the real chunk, and the symbolic link was accidentally deleted.

If the missing file is a symbolic link, you simply need to restore or recreate the file in its original location. The only difficulty part is that Informix doesn't tell you which file it was symbolically linked to. Restoring the symbolic link from your regular filesystem backups is probably the easiest answer. Another method would be to consult any documentation that you may have about how you put the instance together. (Restoring from backup is obviously much easier.)

If it is not a symbolic link, the damaged file may be a filesystem file or raw device. If it is a filesystem file and the filesystem itself is intact, simply recreate a new file with the touch command. After doing so, make sure that the file is read/write for the informix user and informix group. If the filesystem is not intact, you need to relocate the file. Hopefully, you followed the common practice of using symbolic links to point to the actual chunks. If you did, you can recreate the chunk file anywhere on the system and just change the symbolic link to point to the new location. If you did not, you need to make a symbolic link in the original location to point to the new file.

For example, assume that the filesystem /data1 is destroyed, and it contained chunk /data1/rootdbs.dbf. However, you set up the Informix instance to point directly to /data1/rootdbs.dbf, instead of to a symbolic link to that chunk. You create a new file called rootdbs.dbf in /data2, but you have to tell oninit to use the new file. You need to unmount /data1 (although it probably is already) and create a symbolic link in the old location with the following command:

$ ln -s /data2/rootdbs.dbf /data1/rootdbs.dbf This is, of course, a very bad solution since repairing and remounting /data1 will overwrite the symbolic link. If you have to do this, consult your IDS Adminstration Manual about permanently relocating the file. (Use a symbolic link this time.)

Before continuing, you may wish to verify that all chunks are all right. If you don't have a complete list of filenames, you can obtain them by running the strings command on a root dbspace chunk or a ontape archive:

$ zcat /informix/logical/crash/crash.level.1.Z \

| strings | grep '^/'

Make sure that you have checked both of the following conditions:

Permissions

Ensure that someone didn't accidentally change the ownership or permissions of any chunks. If you are using symbolic links to point to the actual chunks, the only permissions that matter are those for the final file to which the symbolic link is pointing. For example, suppose that you have a symbolic link called /informix/chunk1 that points to /dev/dsk/c0t0d0s5. If you are running Solaris, and if you run an ls -l on /dev/dsk/c0t0d0s5, you will find this: lrwxrwxrwx 1 root other 84 Nov 5 02:29 /dev/dsk/c0t0d0s5 -> ../..

/devices/iommu@f,e0000000/sbus@f,e0001000/espdma@f,400000

/esp@f,800000/sd@0,0:f

It is the permissions of sd@0,0:f that matter, not the symbolic link /dev/dsk/c0t0d0s5. To verify its permissions, run the following command: $ Ls -lL /dev/dsk/c0t0d0s5 Adding the -L option causes it to display the ownership and permissions of the source file, not of the link. Make sure that both the owner and group are informix, and that the file is read/write for both the owner and the group.
Symbolic link This problem usually happens when the DBA is trying to "clean house." If you're using symbolic links to point to the actual chunks, and someone deletes the symbolic link, you obviously have a problem. If you are using symbolic links, make sure that you remake the symbolic link instead of creating a chunk in its place. If the instance is currently down because a critical dbspace could not be opened, return to Step 1 and run oninit again. If the instance is already up and you were sent here by Step 8, proceed to Step 9. If you have repaired or replaced any missing chunks, and the instance still won't start, proceed to Step 6. Step 6: Performing a Cold Restore You should be performing this step only if directed to do so by Step 5. Make sure you need this step This is not a step to be taken lightly. Depending on the size of your instance, a cold restore may take a considerable amount of time. Please take a moment to verify that you really do need to perform a restore; if you do, follow the appropriate section below. Can't I just restore the critical dbspaces?

Both ontape and onbar allow you to specify a list of dbspaces to restore. This even works with critical dbspaces. However, if you restore just the critical dbspaces, the restore leaves all other chunks in an "inconsistent" state, as specified by the "I" flag that they display after the restore is done. Informix support does have a tool that will change this flag to consistent, but your mileage will vary on this one. If you are restoring a critical dbspace, you should really restore the whole thing.

Restoring with ontape Make sure to remove the current logical log "tape" prior to beginning a physical restore. If you're backing up to disk, this means moving the file to a different location. If you're backing up to tape, this means physically removing the current logical log tape. When performing a physical restore with ontape, it asks you if you want to back up the current log, and you should always say yes. However, this performs an ontape -a backup, not an ontape -c backup. Remember that the primary difference between these two options is that ontape -a overwrites the "tape" currently in the drive. If it contains any logs other than the current log, they will disappear forever. After removing the current logical log tape or file from its current location, place the latest level 0 archive in the device specified by $TAPEDEV. If you are backing up to disk, this may require uncompressing and moving a file to the appropriate location. If your backing up to tape, it involves placing the latest level 0 archive in the tape drive. After doing so, execute the following command: $ ontape -r Ontape prompts you for everything that it needs. You need to know if you have a level 1 or 2 backup, since it will ask you that. You'll also need to know the location of all logical log backups since the latest archive was taken. Informix asks you if you want to back up the logs. Always say yes, but make sure that you are giving it a fresh tape or file to back up to, since it will overwrite it -- not append to it. The following example was done with an instance that is archiving to disk using infback.sh. Infback.sh now uses named pipes, so the value for $TAPEDEV will be a named pipe (e.g., /informix/logical/crash.level.1.fifo) that reflects the last level that was performed. Since infback.sh will recreate the named pipe when it needs to, we will overwrite it with symbolic links. The example instance also used rclogs.sh to continuously back up the logical logs to disk. The value for $LTAPEDEV is /informix/logical/crash.log, and when it switches logs, it copies them to /informix/logical/$ONCONFIG.year.month.day.hour.minute.second.

An ontape restore with archives on disk always requires more than one window, and this section needs to show both windows below to fully demonstrate the example. To reduce confusion, it uses a regular paragraph like this one when switching windows. Since it still needs to explain the reasoning behind certain commands or answers within a window, it uses this font to do that. There is also a heading on each body of computer output specifying either Restore Window or Alternate Window. The restore window is the window where the ontape -r command is being run, and the alternate window will be the window where we perform other commands. We will start with Figure 13-20, the restore window.

[Restore Window]

#The first thing that we need to do is

#uncompress the archive files

curtis$ uncompress /informix/logical/crash/crash.level.*.Z

curtis$ ls /informix/logical/crash

crash.level.0 crash.level.1

#Now we need to remove the named pipe and replace it with a

#symbolic link

#to the actual backup file

curtis$ rm /informix/crash.level.0.fifo

curtis$ ln -s /informix/logical/crash/crash.level.0 /informix/crash.level.0.fifo

#Now we can begin the restore

curtis$ ontape -r

Please mount tape 1 on /informix/logical/crash/crash.level.0 and press Return to continue ...

Archive Tape Information

Tape type: Archive Backup Tape

Online version: INFORMIX-OnLine Version 7.23.UC4

Archive date: Thu Jan 21 00:57:14 1999

User id: informix

Terminal id: ?

Archive level: 0

Tape device: /informix/crash.level.0.fifo

Tape blocksize (in k): 16

Tape size (in k): 1024000

Tape number in series: 1

Spaces to restore:

1 [rootdbs ]

2 [plogdbs ]

3 [llogdbs ]

4 [testdbs ]

Archive Information

INFORMIX-OnLine Copyright(C) 1986-1995 Informix Software, Inc.

Initialization Time 03/04/98 20:08:25

System Page Size 2048

Version 4

Archive CheckPoint Time 01/21/99 00:57:17

Dbspaces

number flags fchunk nchunks flags owner name

1 2 1 1 M informix rootdbs

2 2 2 1 M informix plogdbs

3 2 3 1 M informix llogdbs

4 1 4 1 N informix testdbs
 
 

Chunks

chk/dbs offset size free bpages flags pathname

1 1 0 10000 9051 PO- /informix/rootdbs.dbf

1 1 0 10000 0 MO- /informix/rootdbs_mirror.dbf

2 2 0 5000 2447 PO- /informix/physlog.dbf

2 2 0 5000 0 MO- /informix/physlog_mirror.dbf

3 3 0 5000 3447 PO- /informix/logiclog.dbf

3 3 0 5000 0 MO- /informix/logiclog_mirror.dbf

4 4 0 500 191 PO- /informix/testdbs.dbf

#Ontape displays all this information to you so that you know

#that this is the right tape to restore the right instance.

#It doesn't actually do

#anything until you respond "y" to the next question.

Continue restore? (y/n)y

#Always say "YES" to this next question.

Do you want to back up the logs? (y/n)y

Please mount tape 1 on /informix/logical/crash.log and press Return to continue ...

Would you like to back up any of logs 65 - 67? (y/n) y

Figure 13-20: Starting an ontape restore
This next section is from another window. We need to move the old logical log "tape" out of the way so that the salvaging of the current log does not overwrite it. In the example in Figure 13-21, we will use the same naming convention as the other files. [Alternate Window]

curtis$ cp crash.log crash.log.1999.01.21.17.04.00

curtis$ compress crash.log.1999.01.21.15.05.16

curtis$ ls -l crash.log.1999.01.21*

total 2424

-rw-rw---- 1 informix informix 73961 Jan 21 01:12 crash.log.1999.01.21.01.13.02.Z

-rw-rw---- 1 informix informix 1949 Jan 21 01:13 crash.log.1999.01.21.01.14.08.Z

-rw-rw---- 1 informix informix 557056 Jan 22 17:04 crash.log.1999.01.22.17:04:00.Z

Figure 13-21: Preparing disk-based logical log backups
Once we've copied the "tape" to another location, it is safe to tell Informix to salvage the current logical log. Note in Figure 13-22 that when it asks for the oldest log that we would like to back up, we answer with the oldest number available. (It never hurts to have too many logical log backups. If we were to answer "66," what would happen if the restore needed log 65 and it had not been backed up, or its backup had been damaged? We would be out of luck, that’s what. [Restore Window]

Logical logs 65 - 67 may be backed up.

Enter the id of the oldest log that you would like to backup? 65

Please label this tape as number 1 in the log tape sequence.

This tape contains the following logical logs:

1 - 67

Log salvage is complete, continuing restore of archive.

#we do have a level one archive, so when it asks if we have one,

#we will answer "yes."

Restore a level 1 archive (y/n) y

Ready for level 1 tape
 
 

Figure 13-22: Backing up the current logical logs
You may recall that prior to beginning the restore, we created a symbolic link from the level 0 archive on disk to the location that Informix expects the archive to be. Now that we are "swapping tapes," we need to remove that link and create another one that points to the level 1 backup. (The commands in Figure 13-23 are obviously being done in another window.) [Alternate Window]

curtis$ rm /informix/crash.level.0.fifo

curtis$ ln -s /informix/logical/crash/crash.level.1 /informix/crash.level.0.fifo

Figure 13-23: Simulating a tape swap
Now that we have swapped tapes, we can respond to the prompt shown in Figure 13-24. [Restore Window]

Please mount tape 1 on /informix/logical/crash/crash.level.0 and press Return to continue ...

Archive Tape Information

Tape type: Archive Backup Tape

Online version: INFORMIX-OnLine Version 7.23.UC4

Archive date: Thu Jan 21 01:10:13 1999

User id: informix

Terminal id: ?

Archive level: 1

Tape device: /informix/crash.level.1.fifo

Tape blocksize (in k): 16

Tape size (in k): 1024000

Tape number in series: 1

#We do not have a level to archive, so we will answer no to

#following prompt.

Restore a level 2 archive (y/n) n

#We do want to restore log tapes, though...

Do you want to restore log tapes? (y/n)y

Roll forward should start with log number 65

Figure 13-24: Responding to ontape’s prompts
Again, we must move over to the other window and prepare the logical log "tape." First, we move the salvage logs to an appropriately named file and compress it. Then we use the log concatenation method discussed in the previous sidebar. Since the logs are compressed, in Figure 13-25 we create a single concatenated log using zcat. [Alternate Window]

curtis$ mv crash.log crash.log.1999.01.21.18.00.00

curtis$ compress crash.log.1999.01.21.18.00.00

curtis$ ls -l crash.log.1999.01.2*

total 2424

-rw-rw---- 1 informix informix 73961 Jan 21 01:12 crash.log.1999.01.21.01.13.02.Z

-rw-rw---- 1 informix informix 1949 Jan 21 01:13 crash.log.1999.01.21.01.14.08.Z

-rw-rw---- 1 informix informix 557056 Jan 22 17:04 crash.log.1999.01.22.17:04:00.Z

-rw-rw---- 1 informix informix 557056 Jan 22 18:00 crash.log.1999.01.22.18.00.00.Z

curtis$ zcat *1999* >crash.log

curtis$ chmod 664 * crash.log

Figure 13-25: Creating one large logical log backup
Now that we have created the single log, we can respond to the following prompt in Figure 13-26. [Restore Window]

Please mount tape 1 on /informix/logical/crash.log and press Return to continue ...

#Since we put all logs into this single log, there are no

#more logs to restore.

Do you want to restore another log tape? (y/n)n

Program over.

#The next step is very important. You must bring the

#instance online when you are done, or you will

#need to do the restore all over again.

curtis$ onmode -m

Figure 13-26: Completing the restore and starting the instance
 
 
 
 
 
 

Make sure that you use onmode -m to bring the instance online after doing a cold restore. If you do not, you will need to completely redo the restore if you stop and start the instance before doing so.

Restoring with onbar

The first and simplest recovery with onbar is to enter onbar -r. This specifies to do a complete restore of any offline dbspaces. It automatically performs the following three steps for you:

  1. onbar -l -s (salvage the logical log).
  2. onbar -r -p [-w] (complete physical restore, which reads the archive only)
  3. onbar -r -l (complete logical restore, which reads all available logical logs)
You may also add an optional -w flag (onbar -r -w) that specifies using the latest whole backup when performing the restore. If you have not been performing -w backups, then you cannot use -w on the restore. If you have been using -w on your backups, then you can use the same option on the restore. You also have the option of not using the -w option on the restore, even if you did use it to back up.

Unlike ontape, you do not even need to move files around or swap tapes if you have an autochanger. It automatically retrieves the appropriate volumes that it needs to write to or read from. Even if you do not have an autochanger, it prompts you for the appropriate tapes by name.

You also have the option of performing the three steps by yourself. This allows you to use a number of flags to do different kinds of restores based on your needs of the moment. The first thing you need to do, though, is issue the onbar -l -s command to salvage any logical logs that have not been backed up.

After doing that, you have a number of options when performing the physical and logical restores. (As started earlier, a physical restore is one that just reads the archive tape. It does not apply any logical logs. Applying the logical logs is called the logical restore. The following onbar command represents your options when beginning the physical restore. Please note the grouping of the options. The -p, -n and -t options are mutually exclusive, and so are the -w, dbspace_list and noflags options.

$ onbar -r \

[ -p | -n xxx | -t time ] [ -w | dbspace_list | noflags ]

Here is a listing of the various options and how they affect the restore:

-p

Adding the -p option to the onbar -r command tells it to perform only the physical restore. If you use this option, you need to run the onbar -r -l command to perform the logical restore. If you do not specify this option, onbar performs both a physical and a logical restore. -n xxx or -t time If you do not specify -p, you can also use these flags to decide how the logical restore is performed. For details on these flags, see the following logical restore section. -w This specifies to use the latest whole backup when restoring the database. Although using this flag will perform a restore of the whole database, this flag is actually telling onbarwhich backup to use, not what kind of restore to do. The reason this flag is here is that if you do restore from a whole backup, you have the option of not doing a logical restore, since you have restored the database to a consistent point in time. (This option is not available to you if you have not been making backups with the -w option.) dbspace_list If you do not use the -w option, you can optionally list the dbspaces that onbar should recover, separated by white space. noflags This is actually just a placeholder to demonstrate what would happen if you used no other flags at all. If you enter onbar -r or onbar -p without specifying -w or a list of dbspaces to recover, it automatically detects and recovers any dbspaces that are offline. The noflags option, as described here, is meant to reiterate the fact that you do not have to specify the -w flag to get a complete restore. The -w flag specifies the restore's source, not its destination. If you specified just a physical restore, you may now perform the logical restore. When doing so, you have three options:

onbar -r -l

This is the default method, and performs a logical restore using all logical logs that were created since the latest archive. onbar -r -l -n lognumber If you know the last log that you wish to use, you may use this option. You may specify the last log that onbar should read using -n lognumber. onbar -r -l -t time You've been waiting for this one. You know that you accidentally deleted a major table at 14:02. You would like to replay all transactions except that one. You may tell onbar to do this using the new -t time option. (In the previous deleted table example, you would probably enter onbar -r -l 14:00.) Make sure that you use onmode -m to bring the instance online after doing a cold restore. If you do not bring the instance online with onmode –m before the next time you stop and start the instance, you will need to completely redo the restore In summary, an onbar restore offers you the same simplicity as an ontape restore, since both have the all-encompassing -r option that means to do a complete physical and logical restore. However, if you need extra options like point-in-time or point-in-log restores, they are available. It also has the added benefit of working with your storage manager so that you no longer have to worry about what tape has what backup. If you have not begun to look at onbar, perhaps now is the time to start. Nine out of every 10 restores will end right here. To make sure that everything is okay, return to step 1 and restart the instance after first bringing it online with onmode -m. Step 7: Are there Errors in the Online Log?

Perhaps the instance started on your first try. Perhaps you needed to do a cold restore in order to get it started. The next thing to do would be to check the online log for any errors. Examples of the types of errors you may see are shown in Figure 13-27:

23:27:34 Assert Failed: WARNING! pthdrpage:ptalloc:bad bfget

23:27:34 Who: Session(7, informix@curtis, 0, 169149316)

Thread(13, fast_rec, a12ccd8, 1)

23:27:34 Results: Cannot use TBLSpace page for TBLSpace 4194305

23:27:34 Action: Run 'oncheck -pt 4194305'

23:27:34 See Also: /tmp/af.d79e5

23:27:34 Cannot Open DBspace 4.

Figure 13-27: Example errors in the online log
If you see any errors like this, you should run an onstat -d to see which chunk is having a problem: Chunks

address chk/dbs offset size free bpages flags pathname

# Output abbreviated...

a12a508 4 4 0 500 191 PD- /informix/testdbs.dbf

The flags above show you that the /informix/testdbs.dbf chunk is down. What you need to find out now is why it is down. If you start the instance and there are no errors in the online log, then proceed to Step 16. If there are errors in the log, proceed to Step 8. Step 8: Is There an Inaccessible Non-Critical Chunk?

If oninit is able to access all critical chunks, it brings the instance online. If any non-critical chunks are inaccessible, it just logs the problem in the online log. If, after checking the online log and running an onstat -d, you have verified that a non-critical chunk is inaccessible to Informix, you need to repair or replace it.

If a non-critical chunk is inaccessible, return to Step 5. If you have verified that the problem chunk is now accessible and has the correct permissions, proceed to Step 10. Step 9: Is There a Corrupted Non-Critical Chunk? You should be performing this step only if directed to do so by Steps 5 or 8. You might not need a restore

The best way to find out if your non-critical chunks are corrupted is to try to bring them online. In order to be able to do that, the following conditions must be true:

If all these conditions are true, you can probably save yourself a restore by bringing the dbspaces online using onspaces. Run the following command for each chunk that was marked down: $ onspaces -s dbspacename -p chunkname -o offset -O

Warning: Bringing chunk back online.

Do you really want to continue? (y/n)y

Verifying physical disk space, please wait ...

Chunk status successfully changed.

If you see successful messages like this one, you won't even need to do a restore. If it complains that the chunk is inconsistent, you have to do a restore to bring it to a consistent state. If you were able to bring all down dbspaces online, proceed to Step 11. If not, proceed to Step 10 to restore them. Step 10. Perform a dbspace Restore

There isn't much that can be said in this step that wasn't already covered in Step 6. However, there are a few differences between the restore discussed in Step 6 and this one:

Read Step 6 in detail, then run one of the following commands: $ ontape -r -D dbspace dbspace#Will recovery any dbspaces

$ onbar -r #Will recover all down dbspaces

$ onbar -r dbspace#Will recover any dbspaces listed

Both onbar and ontape prompt you with the same standard questions. The main differences are that you may be warned that the affected dbspaces will be taken offline, and you will not be asked the "Do you want to back up the logs?" question. A sample output from an ontape restore can be found in Figure 13-28: curtis$ onstat -d|grep testdbs.dbf

a12a508 4 4 0 500 191 PD- /informix/testdbs.dbf

curtis$ ontape -r -D testdbs

DBspace 'testdbs' is online; restoring 'testdbs' will bring all chunks

comprising the DBspace OFFLINE and will terminate all active

transactions and queries accessing the DBspace.

OK to continue?y

Please mount tape 1 on /informix/logical/crash/crash.level.0 and press Return to continue ...

Archive Tape Information

Tape type: Archive Backup Tape

Online version: INFORMIX-OnLine Version 7.23.UC4

Archive date: Thu Jan 21 00:57:14 1999

User id: informix

Terminal id: ?

Archive level: 0

Tape device: /informix/crash.level.0.fifo

Tape blocksize (in k): 16

Tape size (in k): 1024000

Tape number in series: 1

Continue restore? (y/n)y

Spaces to restore:1 [testdbs ]

Restore a level 1 archive (y/n) n

Do you want to restore log tapes? (y/n)y

Roll forward should start with log number 65

Please mount tape 1 on /informix/logical/crash.log and press Return to continue ...

Do you want to restore another log tape? (y/n)n

Program over.

curtis$ onstat -d|grep testdbs.dbf

a12a508 4 4 0 500 191 PO- /informix/testdbs.dbf

Figure 13-28: Sample ontape output
Once the restore of all down dbspaces is complete, they will be brought online. Once all down dbspaces are restored and online, proceed to Step 11. Step 11: Are there Wrong Values in the onconfig File?

If you were forced to use an old onconfig file backup, or create one from scratch, you may have some potentially wrong values. Depending on which values are wrong, they may prevent the instance from operating properly. If so, oninit logs them in the online log.

02:13:58 Onconfig parameter LTAPEBLK modified from 32 to 16.

02:14:46 Onconfig parameter MIRROROFFSET modified from 1 to 0

If you see any errors like this, proceed to Step 12. If not, then proceed to Step 13.
Step 12: Change the Bad Values in the onconfig File

This one is about as easy as they come. Change any bad values in the onconfig file back to their original values. For example, if you saw the errors displayed in Step 11, you need to change LTAPEBLK to 32 and MIRROROFFSET to 1. Unfortunately, most of these values are read only at startup.

Once you have changed any incorrect values, proceed to Step 13. Step 13: Ensuring that the Instance will Restart

If you changed any values in Step 12, you need to restart the instance to have oninit read the new values. Also, depending on the number of steps that you had to follow to get to this step, you may want to make sure that everything will start correctly the next time. The only way to be sure of that is to restart the instance now.

If you wish to restart the instance, proceed to Step 14. If not, proceed to Step 15. Step 14: Taking the Instance Offline

If you had to do any restores to get to this step, make sure that you bring the instance online before you take it offline again. To make sure that it is online, run the following command:

$ oninit -

INFORMIX-OnLine Version 7.23.UC4 -- On-Line -- Up 00:00:29 -- 8976 Kbytes

If you see the above output, then the instance was brought online. If you see the output below, you need to bring the instance online by running the command onmode -m. INFORMIX-OnLine Version 7.23.UC4 -- Quiescent -- Up 00:01:17 -- 8976 Kbytes Once you are sure that the instance is online, you can take it offline: $ onmode -ky Once you have done so, return to Step 1. Step 15: Confirming that dbspaces and Chunks are Online

If you don't restart the database, you should make doubly sure that all dbspaces and chunks are online. To do so, run the command onstat –d, as shown in Figure 13-29:

curtis$ onstat -d

INFORMIX-OnLine Version 7.23.UC4 -- On-Line -- Up 00:06:45 -- 8976 Kbytes

Dbspaces

address number flags fchunk nchunks flags owner name

a12a100 1 2 1 1 M informix rootdbs

a12a790 2 2 2 1 M informix plogdbs

a12a800 3 2 3 1 M informix llogdbs

a12a870 4 1 4 1 N informix testdbs

4 active, 2047 maximum

Chunks

address chk/dbs offset size free bpages flags pathname

a12a170 1 1 0 10000 9307 PO- /informix/rootdbs.dbf

a12a248 1 1 0 10000 0 MO- /informix/rootdbs_mirror.dbf

a12a358 2 2 0 5000 2447 PO- /informix/physlog.dbf

a12a5e0 2 2 0 5000 0 MO- /informix/physlog_mirror.dbf

a12a430 3 3 0 5000 3447 PO- /informix/logiclog.dbf

a12a6b8 3 3 0 5000 0 MO- /informix/logiclog_mirror.dbf

a12a508 4 4 0 500 191 PO- /informix/testdbs.dbf

4 active, 2047 maximum

Figure 13-29: A sample onstat output
Check the flags column in the Dbspaces section for flags P, L, or R. Also check the flags column of the Chunks section for flags D or I. The meanings of these flags are:

P

The dbspace has been physically recovered and is awaiting logical recovery. L The dbspace is being logically recovered. R The dbspace is being physically recovered. D The chunk is down. I The chunk is in an inconsistent state. Once you know if there are any of these flags, proceed to step 18. Step 16: Recovering a Deleted Table or Database

Perhaps the instance started OK, but there is a different problem. If a DBA accidentally deleted a dbspace, or a user accidentally deleted an important table, there is really only one way to recovery that -- a point-in-time restore.

If you need a point-in-time restore, proceed to Step 17. If not, proceed to Step 18. Step 17: Performing a Point-in-Time Restore

In order to do a point-in-time restore, you need to do a cold restore of the entire database. (Details on how to do that are in Step 6.)

If you are using ontape, you will need to apply all logical logs until you reach the one during which the user/DBA error occurred. Do not apply that logical log.

If you are using onbar, you can use the -n xxx or -t time features of onbar -r to recover up to a point in time just prior to the user/DBA error.

Once you have done this, proceed to Step 19. Step 18: Is Everything OK? If you saw any of the flags mentioned in Step 15, return to Step 8. If not, proceed to Step 19. Step 19: Making a Backup

Every restore should be followed immediately by a full backup. Of course, Informix allows you do so online. Don’t consider the restore finished until you have completed this backup.