new user (memo field limit)

[shibu ninan]

hi,
I am a new monarch user (using monarch 7). given below is the layout of my report. the 'end of report is my detail' , I have an append template for the header, and it works since the line numbers are fixed. my problem is the actual report body has different headings based on the type of report. So I cannot have a fixed template. I have 2 questions.
1. can I select the entire body of my report into one field (monarch giving me restrictions after 64lines).
2. I have the word 'diagnosis' on every report. on some reports its 'descriptive diagnosis'. I have an append multiline template starting from diagnosis to the beginning of the next template (the 'endofreport' detail template). how do I tell the template to select it no matter whether its 'diagnosis' or 'descriptive diagnosis'.


Hosptial Name Pt. Name: ABCDEF, ABCDEFGHIJ
LABORATORY SERVICES (00000)999999999
Top Fanin st Att. Phys: ABCDEF, ABCDEF A
Houston, Texas 77444-1501 Ord. Phys: ABCDEF, ABCDEF A
(713) 604-4327 Cons.Phys: ABCDEF, ABCDEF A

Client: abcdefg HOSPITAL DOB: 01/01/1999 Age: 11 YRS Sex: M

Collected Date:XX/XX/XX XXXX Received Date: XX/XX/XX 1051
Case Number: HS-02-99999
__________________________________________________ ____________________________________________
S U R G I C A L P A T H O L O G Y_____________________________


CLINICAL INFORMATION
Preoperative diagnosis: xxxxxxxxxxxxxxxxxxxx

TISSUE/SOURCE DESCRIPTION
"xxxxxxxxxxxxxx biopsy"

GROSS DESCRIPTION
The specimen is received in formalin in a container labeled with the patient's
name, medical record number and "xxxxxxxxxxx".

MICROSCOPIC DESCRIPTION
The GDFGFDGFDGG FGDFF is preserved. There is very mild and focal portal
lymphocytic infiltrate with focal DFGDFGDF DFG DFGFDG DFGDFGFDGDF.

DIAGNOSIS
xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade .
Stage 0.


Code 1
CPT# XXXXX

By: 99999999999999, M.D.
09/24/03 (Electronic Signature)
SDFSDFSDFSDFSDFS - Houston
SDFSDFSDFSDFSDF


End of Report







Hosptial Name Pt. Name: ABCDEF, ABCDEFGHIJ
LABORATORY SERVICES (00000)999999999
Top Fanin st Att. Phys: ABCDEF, ABCDEF A
Houston, Texas 77444-1501 Ord. Phys: ABCDEF, ABCDEF A
(713) 604-4327 Cons.Phys: ABCDEF, ABCDEF A

Client: abcdefg HOSPITAL DOB: 01/01/1999 Age: 11 YRS Sex: M

Collected Date:XX/XX/XX XXXX Received Date: XX/XX/XX 1051
Case Number: CG-02-99999
__________________________________________________ ____________________________________________
CYTOLOGY REPORT ____________________________


CLINICAL INFORMATION
Preoperative diagnosis: xxxxxxxxxx.

TISSUE/SOURCE DESCRIPTION
"xxxxxxxxxxxxxxx"

GROSS DESCRIPTION
The specimen is received in formalin in a container labeled with the patient's
name, medical record number and "liver biopsy".

MICROSCOPIC DESCRIPTION
The GDFGFDGFDGG FGDFF is preserved. There is very mild and focal portal
lymphocytic infiltrate with focal DFGDFGDF DFG DFGFDG DFGDFGFDGDF.

DESCRIPTIVE DIAGNOSIS
xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade a-c/d.
Stage 0.


Code 1
CPT# XXXXX

By: 99999999999999, M.D.
09/24/03 (Electronic Signature)
SDFSDFSDFSDFSDFS - Houston
SDFSDFSDFSDFSDF


End of Report Any help would be greatly appreciated. thanks,

[Grant Perkins]

Interesting questions.

Instant response without trying anything is as follows;

1. I would say so though I would be tempted to select in 2 section using "Clinical Information" as the start of each section and the section header above each as an append somehow. (Have not yet checked out the best way to get to that).

2. V7 feature that allows

"Start Field on String "xxxxxxxxx" anywhere in previous line"

should get you there? Then just select the entire width of report. The word diagnosis anywhere in the previous line will be enough.

I hope this helps.

Best regards,

Grant



Originally posted by shibu ninan:
hi,
I am a new monarch user (using monarch 7). given below is the layout of my report. the 'end of report is my detail' , I have an append template for the header, and it works since the line numbers are fixed. my problem is the actual report body has different headings based on the type of report. So I cannot have a fixed template. I have 2 questions.
1. can I select the entire body of my report into one field (monarch giving me restrictions after 64lines).
2. I have the word 'diagnosis' on every report. on some reports its 'descriptive diagnosis'. I have an append multiline template starting from diagnosis to the beginning of the next template (the 'endofreport' detail template). how do I tell the template to select it no matter whether its 'diagnosis' or 'descriptive diagnosis'.


Hosptial Name Pt. Name: ABCDEF, ABCDEFGHIJ
LABORATORY SERVICES (00000)999999999
Top Fanin st Att. Phys: ABCDEF, ABCDEF A
Houston, Texas 77444-1501 Ord. Phys: ABCDEF, ABCDEF A
(713) 604-4327 Cons.Phys: ABCDEF, ABCDEF A

Client: abcdefg HOSPITAL DOB: 01/01/1999 Age: 11 YRS Sex: M

Collected Date:XX/XX/XX XXXX Received Date: XX/XX/XX 1051
Case Number: HS-02-99999
__________________________________________________ ____________________________________________
S U R G I C A L P A T H O L O G Y_____________________________


CLINICAL INFORMATION
Preoperative diagnosis: xxxxxxxxxxxxxxxxxxxx

TISSUE/SOURCE DESCRIPTION
"xxxxxxxxxxxxxx biopsy"

GROSS DESCRIPTION
The specimen is received in formalin in a container labeled with the patient's
name, medical record number and "xxxxxxxxxxx".

MICROSCOPIC DESCRIPTION
The GDFGFDGFDGG FGDFF is preserved. There is very mild and focal portal
lymphocytic infiltrate with focal DFGDFGDF DFG DFGFDG DFGDFGFDGDF.

DIAGNOSIS
xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade .
Stage 0.


Code 1
CPT# XXXXX

By: 99999999999999, M.D.
09/24/03 (Electronic Signature)
SDFSDFSDFSDFSDFS - Houston
SDFSDFSDFSDFSDF


End of Report







Hosptial Name Pt. Name: ABCDEF, ABCDEFGHIJ
LABORATORY SERVICES (00000)999999999
Top Fanin st Att. Phys: ABCDEF, ABCDEF A
Houston, Texas 77444-1501 Ord. Phys: ABCDEF, ABCDEF A
(713) 604-4327 Cons.Phys: ABCDEF, ABCDEF A

Client: abcdefg HOSPITAL DOB: 01/01/1999 Age: 11 YRS Sex: M

Collected Date:XX/XX/XX XXXX Received Date: XX/XX/XX 1051
Case Number: CG-02-99999
__________________________________________________ ____________________________________________
CYTOLOGY REPORT ____________________________


CLINICAL INFORMATION
Preoperative diagnosis: xxxxxxxxxx.

TISSUE/SOURCE DESCRIPTION
"xxxxxxxxxxxxxxx"

GROSS DESCRIPTION
The specimen is received in formalin in a container labeled with the patient's
name, medical record number and "liver biopsy".

MICROSCOPIC DESCRIPTION
The GDFGFDGFDGG FGDFF is preserved. There is very mild and focal portal
lymphocytic infiltrate with focal DFGDFGDF DFG DFGFDG DFGDFGFDGDF.

DESCRIPTIVE DIAGNOSIS
xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade a-c/d.
Stage 0.


Code 1
CPT# XXXXX

By: 99999999999999, M.D.
09/24/03 (Electronic Signature)
SDFSDFSDFSDFSDFS - Houston
SDFSDFSDFSDFSDF


End of Report Any help would be greatly appreciated. thanks,

[shibu ninan]

when I tried to put 'diagnosis' as the string xxxxxxx in the previous line, monarch still needs something on the trap line. Am I doing something wrong. I was trying to create a memo multiline field starting from 'diagnosis' to the end of the report.

[shibu ninan]

sorry about the earlier post. I did get the diagnosis thing to work (the string xxx in the previous line is a life saver.

can somebody also help me on how to capture the entire body. the headers are not always the same for eg: the report might not have a 'clinical information header'. It might have only a gross header and a diagnosis header. basically I want to be able to say: give me everything in between the header and the end of report into one field.
thanks again

[Grant Perkins]

Thinking 'aloud' again I was wondering if you could grab each section under, for example, S U R G I C A L P A T H O L O G Y or CYTOLOGY REPORT as a complete text block (if that is what you want to do).

The lines around those headings look like they may offer some trapping facilities but I never feel certain based on looking at a cut and paste version of a report - they can appear to be things they are not!!

If that fails try trapping on "Case Number:" and start a multi line field 3 or so lines below that using 4/5 line template according to need. I hope that the records do not extend over more than one page OR that if they do they don't have another full header section on page 2.

If there are other problems after that things may start to get a little interesting :cool:

The whole page header section can be either a page header or ordinary append I would guess.

"end of report" looks like it would be best set up as a footer.

See if any of those ideas work for you and let us know what happens. I may have a chance to look at your sample over the weekend - just for fun of course!

Best regards,

Grant

Addendum for clarity:

I would (I think) see the body of the report as the detail template, the header as either Page header or a simple append and the 'end of report' as a footer marker to terminate the body selection.

Does that make any sense as a description?


Originally posted by shibu ninan:
sorry about the earlier post. I did get the diagnosis thing to work (the string xxx in the previous line is a life saver.

can somebody also help me on how to capture the entire body. the headers are not always the same for eg: the report might not have a 'clinical information header'. It might have only a gross header and a diagnosis header. basically I want to be able to say: give me everything in between the header and the end of report into one field.
thanks again

[ September 26, 2003, 03:41 PM: Message edited by: Grant Perkins ]

[Grant Perkins]

OK, found the time already.

The whole body can be selected in one memo field but some of it looks a bit messy, especially if you want to split it back out again. Slice and dice may be possibly but could be a lot of work depending upon your requirements.

However, if you do back to your original idea of the detail being at the bottom of the report (anything else usable instead of "End of report"?) then each section above could be captured as an append with the page header set as a header I THINK.

Providing you will never find two 'headers' with the same description AND you don't exceed the maximum number of templates possible, then for each possible sub heading description -


CLINICAL INFORMATION

DIAGNOSIS

GROSS DESCRIPTION

and so on

you can create a separate template to capture anything that might appear under the heading. Select using the descriptions and then star a field one row down and as wide as required. Only 'paint' one row. Then go into the field ptions and set "End Row on" to ether a "non blank preceeding string" value or "none of the above" depending upon which works the best.

The "Start field on" string on previous row option may also come into play but I don't think you will need it unless there are more possibilities than available template definitions, in which case things start to get very interesting indeed.

Once again, I hope you can make some sense of this description.

Best regards,

Grant



Originally posted by Grant Perkins:
Thinking 'aloud' again I was wondering if you could grab each section under, for example, S U R G I C A L P A T H O L O G Y or CYTOLOGY REPORT as a complete text block (if that is what you want to do).

The lines around those headings look like they may offer some trapping facilities but I never feel certain based on looking at a cut and paste version of a report - they can appear to be things they are not!!

If that fails try trapping on "Case Number:" and start a multi line field 3 or so lines below that using 4/5 line template according to need. I hope that the records do not extend over more than one page OR that if they do they don't have another full header section on page 2.

If there are other problems after that things may start to get a little interesting :cool:

The whole page header section can be either a page header or ordinary append I would guess.

"end of report" looks like it would be best set up as a footer.

See if any of those ideas work for you and let us know what happens. I may have a chance to look at your sample over the weekend - just for fun of course!

Best regards,

Grant

Addendum for clarity:

I would (I think) see the body of the report as the detail template, the header as either Page header or a simple append and the 'end of report' as a footer marker to terminate the body selection.

Does that make any sense as a description?Edit - corrected ubb code and revised for (hopefully) better clarity.

[ September 28, 2003, 07:04 AM: Message edited by: Grant Perkins ]

[shibu ninan]

thanks a lot for the detailed reply. Like you said, I have now used the header , body, footer template. The body I split into 2 memo fields, everything below diagnosis as one memo , diagnosis is a required field on the report now. everything below casenumber till diagnosis as another memo field.
The reporting people are trying to get the duplicate headers out of second pages if its the same casenumber. that should take care of my extraction problem.
I have a new problem that I overlooked before. the line breaks/carriage returns of my .txt file is not being kept by the monarch. The spaces are intact. for eg:
DIAGNOSIS xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade .
Stage 0


BECOMES
DIAGNOSIS xxxxx, needle biopsy AABCDJLSEIFLSJDFL SDF Grade . Stage 0


any thoughts?. thanks again

[Grant Perkins]

Nice new challenge!

What do you want it to look like?

Do you want separate database fields, one for each line?

Or do you want to view it a block of text formatted as it was in the original?

Or perhaps something else completely different?

It the extraction to be exported to another application, such as Excel, or will it simply be printed into a different format?

I'm not saying there is an answer, just trying get a fix on the requirement.

Grant



Originally posted by shibu ninan:
thanks a lot for the detailed reply. Like you said, I have now used the header , body, footer template. The body I split into 2 memo fields, everything below diagnosis as one memo , diagnosis is a required field on the report now. everything below casenumber till diagnosis as another memo field.
The reporting people are trying to get the duplicate headers out of second pages if its the same casenumber. that should take care of my extraction problem.
I have a new problem that I overlooked before. the line breaks/carriage returns of my .txt file is not being kept by the monarch. The spaces are intact. for eg:
DIAGNOSIS xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade .
Stage 0


BECOMES
DIAGNOSIS xxxxx, needle biopsy AABCDJLSEIFLSJDFL SDF Grade . Stage 0


any thoughts?. thanks again

[shibu ninan]

I would like to view it as a block of text formatted as it was in the original?

Eventually the extracted data from monarch will be shown on the web page as a report combined with some images. thanks

[Grant Perkins]

Hmm, plenty of challenges.

This thread (http://mails.datawatch.com/cgi-bin/ultimatebb.cgi?ubb=get_topic;f=1;t=000099#000000) touches on the same sort of functionality and I suspect that V6.01 would give you a solution!

However we have to deal with V7.

There seem to be two lines to investigate.

If the sections you need to report as text always have the same number of lines with maybe a one line tolerance - for example if usually 4 lines but sometimes 5 - it might be possible to define 5 separate fields in the append template rather than a multi-line field. The smply format the output based on the required number of separate fields.

If the number of lines are too variable for that to work, then we need to find a way to break up the lines that are concatenated by the multi line field concept.

Interestingly it looks like the second line and subsequent lines in the block are indented a little. If that is consistent there is a possibility to try to split the string up into separate calculated fields based on working with the blocks of 3 spaces in the rest of the string. Maybe.

However these are text fields and so may not be consistently keyed in by the operators, so that is a little suspect as a reliable concept.

As an example it is possible to use the EXTRACT function to grab the text between the two '3 space' gaps.

Clearly there are difficulties trying to develop something along those lines from limited samples and it may not be worth going that way if you check the concept against a few more examples and find data format anomalies.

You mentioned that you are expecting to get the report headers modified when multiple pages appear. If you have access to programming resource to change the reports, would it be possible to do something to include a clear line break identifier in the problem text fields? The identifier could then be used as a line split character ready for using Lsplit or Rsplit or similar.

Those are my initial thoughts. However you pose an interesting challenge which I will continue to investigate!

If anyone else already has the solution - please let us know! :confused:

Grant



Originally posted by shibu ninan:
I would like to view it as a block of text formatted as it was in the original?

Eventually the extracted data from monarch will be shown on the web page as a report combined with some images. thanks

[ May 01, 2006, 05:39 PM: Message edited by: Todd Niemi ]

[Grant Perkins]

OK, here goes with one approach which should work for these samples ...

Splitting and re-formatting a multi-line text field using Version 7.

The first thing to look for is some identifier we can use to separate the rows of text one from another. This may not always be possible! However some tricks may be employed to help - like making the fields over sized to the LEFT of the text block if possible, thus introducing a known number of spaces to the 'trimmed' text. As they are left of the main text they will be captured.

In this example we are lucky (it seems) in that lines 2 and onwards from the text block include 2 spaces at the left. When captured this will provide a 3 space 'string' at various points in the text string. It's not foolproof but gives some possibilities.

What we need to do is measure or otherwise find where these break points are and add something at those points to allow us to identify the original line breaks. (LSplit would be great in principle but we don't (yet) have a single character to use to make the split or indeed any characters anywhere that represent the original break points.)

If we can identify the break points we can replace one of the 3 spaces (the one that is added during the extract, so leaving the other 2 that we extracted) with something we can use to split the field back into its separate lines. If we make this a <Carriage return> we can even format the field back to display as per the report when viewed within Monarch if the row height settings are appropriate. There are probably a number of approaches to this problem. The following is where I got to.

Let's assume the field is named "Desc Diagnosis" for this example. and the data has extracted in this format;


xxxxx, needle biopsy: AABCDJLSEIFLSJDFL SDF Grade a-c/d. Stage 0.One approach would be to work out any required field beginning and end points for each line we require. This would give a numerical "modelling" of the line which we could then call on using calculated fields in order to separate the one field into four individual fields using slice and dice techniques. The four fields can then be formatted as required in the final output.

In this example I decided to re-format the field in a series of steps by introducing a <CR> character ( chr(10) ) as the various break points are identified. If you don't want the reformat solution just insert any other suitable character ( | vertical bar for example) that can be picked out easily at a later stage if required.

This particular approach uses the new to Version 7 STUFF function.

Here is the first break point identified;


Stuff([Desc Diagnosis],Instr(" ",[Desc Diagnosis]),1,chr(10))Interpretation:

Use the STUFF function on the multi-line field [Desc Diagnosis], at the position given by the INSTR function that identifies the START of the 3 space gap in [Desc Diagnosis]. Replace the (first) character at that position (the space added by the extraction) with chr(10) (a carriage return or 'new line' character which is not normally visible on screen).

OK so far. Our new field, which I will call "Stuff at end of Line 1" looks like this;



xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF Grade a-c/d. Stage 0. The next stage is to break at the next line. This will give a field I will call "Stuff at end of Line 2"


Stuff([Stuff at end of line 1],instr(Extract([Desc Diagnosis]," "," "),
[Desc Diagnosis])+Len(Extract([Desc Diagnosis]," "," ")),1,chr(10)) Looks complicated? Well, yes it probably is more complicated than is necessary. In fact the protype used discrete fields to make it easier to read but I have replaced the simple field names of the calculated 'work' fields with the formulae that created them. In fact this formula is based on the numerical dissection of the data field, an approach that may be required in other circumstances. And if anyone can interpret it for me would they please let me know how it works. ;)

I thought "nested if " statements were bad enough but nested Stuffs! Where's the coffee and the headache tablets?

On the other hand all we really need to do is repeat the first formula with a change of field name since we have already replaced the 3 spaces with "<CR> " so our INSTR function will now pick up the second "3 spaces" string.


Stuff([Stuff at end of line 1],Instr(" ",[Stuff at end of line 1]),1,chr(10))Our text will now look like this;


STUFF AT END OF LINE 2
xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade a-c/d. Stage 0. You have probably guessed the last calculation we need for this particular 4 section split;


Stuff([Stuff at end of line 2],Instr(" ",[Stuff at end of line 2]),1,chr(10))and that's it.


STUFF AT END OF LINE 3
xxxxx, needle biopsy:
AABCDJLSEIFLSJDFL SDF
Grade a-c/d.
Stage 0. Just hide all the work fields and export the rest.

Hope it makes sense.

Grant



Originally posted by shibu ninan:
I would like to view it as a block of text formatted as it was in the original?

Eventually the extracted data from monarch will be shown on the web page as a report combined with some images. thanks

[shibu ninan]

wow.
I did catch on to the calculations, very hard for me. sorry I did not respond yesterday. i was trying to talk to the hospitals cerner programmer to see if he can insert any special character for line breaks. they won't be able to do it since its a common report going to different places.
I was hoping there was an easier way like using the prep tool, since the diagnosis paragraph could be any number of lines. I was going through that version 6.01 thread and if monarch stores that line break as CrLf (chr(13)+chr(10)), then maybe I can look for that.
Anyways thanks a lot grant, you have been really helpful.

[Grant Perkins]

Hi,

I also thought of using Prep or more likely MSRP but could not quite see a good way that would work as part of the standard report.

If you were to pre-process by extracting parts of the report, Prep or MSRP them, and then combining everything again it might be a useful tool - but hopefully you would not need to do that with any report except as a last resort.

The V6.01 stuff was interesting and (I found) useful in V6. But it is not the way V7 works. However it did provide the idea to insert the <linefeed>, after which it is just a matter of working out how and where. ;)

Provided that you can identify a consistent pattern, or combination of patterns, in a multi line extract it should be possible to split the field using one technique or another and then apply the 'cascading' break points as required.

If the single field needs to be broken up into more than one 'block field' for different splitting concepts to be applied that is not reall a problem either. At worst case you would need to break everything out in any way possible and the concatenate the components with a suitable splitting 'character' applied between each field section as part of the concatenation. Indeed I think before V7, with the exception of V6.01, the break and concatenate process would be the only realistic way to address this. Anyone out there got a ready prepared solution for earlier versions?

I have not looked at how to deal with the possibility of, say, a 20 line field with only 4 lines required, but I reckon you will be able to work out what do to get the final output you require. Checking whether another line is there to be processed (even from Line 1) should really be part of the formula I guess.

I hope you will add to the post if any other awkward problems arise since this one looks like it may still offer some interesting challenges!

All the best,

Grant



Originally posted by shibu ninan:


<snip>

I was hoping there was an easier way like using the prep tool, since the diagnosis paragraph could be any number of lines. I was going through that version 6.01 thread and if monarch stores that line break as CrLf (chr(13)+chr(10)), then maybe I can look for that.
Anyways thanks a lot grant, you have been really helpful.

[shibu ninan]

this is what I ended up doing. I am using monarch to extract the header information from the report file and inserting into microsoft access. then I am using asp.net's regular expression parsing scripts to take the full body portion and the case number, patients name(its easy to weed out the header pages since its consistent) and I am updating the same database with a added memo field where the patient name and case number matches.
It involves 2 more clicks for me overall , but its easier than learning monarch coding. I already knew scripting language perl, so this was not very hard to do. anyways thanks grant and everybody else. have a nice weekend