RL Vision Knowledge Base
Support questions and answers for software by RL Vision.
Note: This is an archived discussion. Any bug, problem or suggestion mentioned here is likely to have been fixed since it was written.
Subject: Re: Duplifind Script please
Date: Mon, 24 Mar 2008 17:31:05 +0100Hi,
The best thing I can do is a script that would allow you to choose which of the "fields" to include in the comparison. I would split each line into fields, based on the commas and colons. Then you could for example remove the "seconds field". Would that be ok?
// Dan
ernie wrote:
> Hello Dan,
>
> I am a registered owner of Dupli Find and I want to first, thank you for such a useful and robust utility, and second ask for a script.
>
> How much for a script that allows a line of text to be matched as a duplicate even if only one or two or three or four or five or six or eight characters are different?
>
> Here is a sample of lines I am trying to make unique:
>
> AC#1,N38152,92%,8HR,VFR,03:11:40,06:18:35, 65 ,F,8270,EGNT,07:11:33,10:18:28, 75 ,F,8269,EDDN
> AC#1,N38152,92%,8HR,VFR,03:11:40,06:18:35, 65 ,F,8270,EGNT,07:11:36,10:18:31, 75 ,F,8269,EDDN
>
> If you look at the times in front of the 75 you will see that the times are different by three seconds and four characters. I would like to set a switch to choose whether one or, up to 8, characters are different. In this case, I consider both these lines as duplicates even thought there are a few differences. I would set the switch to four characters and would want Dupli Find to consider any lines with four different characters in the same position to be considered a duplicate so I can select and delete the duplicates... I hope that is clear :)
>
> Thank you for a fine program and your consideration in advance.
>
> Ed Batres
>
> More sample lines, there should be only thirty-one unique lines:
>
> AC#1,N39112,97%,12HR,IFR,02:28:48,07:16:21, 60 ,F,0241,KFLL,08:28:43,13:16:16, 70 ,F,0242,TJSJ
> AC#1,N39216,64%,12HR,IFR,04:41:26,07:31:09, 80 ,F,6432,KPWK,10:41:15,13:30:58, 90 ,F,6431,KHEF
> AC#1,N39216,64%,12HR,IFR,04:41:26,07:31:09, 80 ,F,6432,KPWK,10:41:20,13:31:03, 90 ,F,6431,KHEF
> AC#1,N39336,26%,12HR,IFR,02:26:31,07:04:40, 110 ,F,3163,OMAD,08:26:26,13:04:35, 120 ,F,3164,OYAA
>
> AC#1,N39368,82%,6HR,VFR,00:25:40,02:12:30, 115 ,F,7827,RPVM,03:25:40,05:12:30, 125 ,F,7828,RPLL
> AC#1,N39368,82%,6HR,VFR,00:25:40,02:12:30, 115 ,F,7827,RPVM,03:25:42,05:12:32, 125 ,F,7828,RPLL note how these three should be dupes
> AC#1,N39368,82%,6HR,VFR,00:25:42,02:12:32, 115 ,F,7827,RPVM,03:25:45,05:12:35, 125 ,F,7828,RPLL
>
> AC#1,N39416,65%,12HR,IFR,05:04:22,08:37:13, 60 ,F,2365,KTVF,11:04:38,14:37:29, 50 ,F,2366,KMIE
> AC#1,N39416,65%,12HR,IFR,05:04:27,08:37:18, 60 ,F,2365,KTVF,11:04:38,14:37:29, 50 ,F,2366,KMIE
> AC#1,N39416,65%,12HR,IFR,05:04:32,08:37:23, 60 ,F,2365,KTVF,11:04:38,14:37:29, 50 ,F,2366,KMIE
> AC#1,N39448,66%,12HR,VFR,04:31:40,07:39:16, 125 ,F,2858,VAJJ,10:31:35,13:39:11, 115 ,F,2859,VOCI
> AC#1,N39456,80%,8HR,IFR,00:16:28,02:46:59, 90 ,F,7893,YGLI,04:16:35,06:47:06, 80 ,F,7894,YMAY
> AC#1,N39456,80%,8HR,IFR,00:16:31,02:47:02, 90 ,F,7893,YGLI,04:16:35,06:47:06, 80 ,F,7894,YMAY
> AC#1,N39552,34%,8HR,IFR,02:15:53,03:58:39, 110 ,F,6086,KRYN,06:15:46,07:58:32, 120 ,F,6085,61B
> AC#1,N39552,34%,8HR,IFR,02:15:53,03:58:39, 110 ,F,6086,KRYN,06:15:49,07:58:35, 120 ,F,6085,61B
> AC#1,N39568,24%,24HR,IFR,00:52:55,06:35:14, 100 ,F,7790,KGCC,12:52:55,18:35:14, 110 ,F,7789,KHTS
> AC#1,N39576,75%,8HR,IFR,00:58:50,01:56:28, 90 ,F,2870,LGMT,04:58:50,05:56:28, 100 ,F,2869,LGAV
> AC#1,N39576,75%,8HR,IFR,00:58:53,01:56:31, 90 ,F,2870,LGMT,04:58:50,05:56:28, 100 ,F,2869,LGAV
> AC#1,N3960,81%,8HR,VFR,00:47:21,03:03:49, 75 ,F,6969,LKKU,04:47:14,07:03:42, 85 ,F,6968,EDDW
> AC#1,N3960,81%,8HR,VFR,00:47:21,03:03:49, 75 ,F,6969,LKKU,04:47:17,07:03:45, 85 ,F,6968,EDDW
> AC#1,N39656,20%,12HR,IFR,04:57:04,10:02:18, 90 ,F,3957,LIRN,10:57:15,16:02:29, 100 ,F,3956,HLGT
> AC#1,N39656,80%,24HR,IFR,00:10:33,02:18:18, 110 ,F,9911,SBES,12:10:12,14:17:57, 100 ,F,9910,SBUR
> AC#1,N39656,80%,24HR,IFR,00:10:33,02:18:18, 110 ,F,9911,SBES,12:10:22,14:18:07, 100 ,F,9910,SBUR
> AC#1,N3984,24%,12HR,IFR,01:49:10,06:24:49, 80 ,F,9723,UHMA,07:48:59,12:24:38, 90 ,F,9724,PAFA
> AC#1,N39912,78%,12HR,IFR,04:55:19,07:42:17, 80 ,F,5352,KFSM,10:55:19,13:42:17, 90 ,F,5351,KPWK
> AC#1,N39912,78%,12HR,IFR,04:55:24,07:42:22, 80 ,F,5352,KFSM,10:55:19,13:42:17, 90 ,F,5351,KPWK
> AC#1,N39944,62%,8HR,VFR,02:02:21,05:15:12, 55 ,F,1467,KPYM,06:02:21,09:15:12, 65 ,F,1466,KCPC
> AC#1,N39944,62%,8HR,VFR,02:02:24,05:15:15, 55 ,F,1467,KPYM,06:02:21,09:15:12, 65 ,F,1466,KCPC
> AC#1,N39952,30%,6HR,IFR,00:22:01,02:33:26, 120 ,F,0785,MHTG,03:22:04,05:33:29, 110 ,F,0784,MMCZ
> AC#1,N40008,52%,12HR,IFR,01:38:37,04:43:46, 110 ,F,8274,LGKA,07:38:37,10:43:46, 120 ,F,8273,LIEE
> AC#1,N40008,52%,12HR,IFR,01:38:42,04:43:51, 110 ,F,8274,LGKA,07:38:37,10:43:46, 120 ,F,8273,LIEE
> AC#1,N40128,7%,24HR,IFR,05:41:11,10:25:31, 60 ,F,1008,KSTP,17:41:01,22:25:21, 70 ,F,1009,KFRG
> AC#1,N40160,48%,6HR,VFR,02:28:27,04:26:57, 75 ,F,7104,KALN,05:28:22,07:26:52, 65 ,F,7105,KBEC
> AC#1,N40224,48%,8HR,IFR,00:29:53,03:43:00, 60 ,F,3925,KHUM,04:29:53,07:43:00, 50 ,F,3926,MUHA
> AC#1,N40408,6%,24HR,IFR,08:41:01,13:47:23, 80 ,F,7929,YBTL,20:41:01,01:47:23, 90 ,F,7930,YSCB
> AC#1,N40448,26%,6HR,IFR,00:54:35,03:08:37, 90 ,F,8121,ESOK,03:54:35,06:08:37, 80 ,F,8122,EDWI
> AC#1,N40464,83%,12HR,VFR,02:40:35,06:12:00, 75 ,F,5207,KMCB,08:40:40,12:12:05, 85 ,F,5208,KHOB
> AC#1,N40528,71%,6HR,IFR,00:56:02,01:34:29, 50 ,F,4694,3T1,03:56:02,04:34:29, 40 ,F,4695,KDFW
> AC#1,N40528,71%,6HR,IFR,00:56:02,01:34:29, 50 ,F,4694,3T1,03:56:04,04:34:31, 40 ,F,4695,KDFW
> AC#1,N40528,71%,6HR,IFR,00:56:04,01:34:31, 50 ,F,4694,3T1,03:56:07,04:34:34, 40 ,F,4695,KDFW
> AC#1,N40576,93%,8HR,IFR,02:43:15,05:57:46, 60 ,F,8550,Q24,06:43:15,09:57:46, 70 ,F,8551,KBTR
> AC#1,N40576,93%,8HR,IFR,02:43:15,05:57:46, 60 ,F,8550,Q24,06:43:18,09:57:49, 70 ,F,8551,KBTR
> AC#1,N40680,18%,6HR,IFR,02:15:24,04:17:20, 110 ,F,2025,MTPP,05:15:18,07:17:14, 120 ,F,2026,MUCM
> AC#1,N40712,12%,8HR,IFR,00:28:08,01:52:21, 120 ,F,0841,KEUG,04:28:08,05:52:21, 110 ,F,0842,KPAE
> AC#1,N40800,25%,12HR,VFR,05:47:10,10:35:38, 115 ,F,9694,OMAL,11:47:05,16:35:33, 125 ,F,9695,OYAA
> AC#1,N40800,61%,8HR,IFR,02:23:02,04:04:03, 120 ,F,8933,KSCK,06:22:55,08:03:56, 110 ,F,8934,KLGB
> AC#1,N40816,58%,12HR,IFR,00:56:52,05:34:40, 80 ,F,0160,KTYR,06:56:57,11:34:45, 90 ,F,0159,KBCT
>
> Windows Live Hotmail is giving away Zunes. Enter for your chance to win.
The best thing I can do is a script that would allow you to choose which of the "fields" to include in the comparison. I would split each line into fields, based on the commas and colons. Then you could for example remove the "seconds field". Would that be ok?
// Dan
ernie wrote:
> Hello Dan,
>
> I am a registered owner of Dupli Find and I want to first, thank you for such a useful and robust utility, and second ask for a script.
>
> How much for a script that allows a line of text to be matched as a duplicate even if only one or two or three or four or five or six or eight characters are different?
>
> Here is a sample of lines I am trying to make unique:
>
> AC#1,N38152,92%,8HR,VFR,03:11:40,06:18:35, 65 ,F,8270,EGNT,07:11:33,10:18:28, 75 ,F,8269,EDDN
> AC#1,N38152,92%,8HR,VFR,03:11:40,06:18:35, 65 ,F,8270,EGNT,07:11:36,10:18:31, 75 ,F,8269,EDDN
>
> If you look at the times in front of the 75 you will see that the times are different by three seconds and four characters. I would like to set a switch to choose whether one or, up to 8, characters are different. In this case, I consider both these lines as duplicates even thought there are a few differences. I would set the switch to four characters and would want Dupli Find to consider any lines with four different characters in the same position to be considered a duplicate so I can select and delete the duplicates... I hope that is clear :)
>
> Thank you for a fine program and your consideration in advance.
>
> Ed Batres
>
> More sample lines, there should be only thirty-one unique lines:
>
> AC#1,N39112,97%,12HR,IFR,02:28:48,07:16:21, 60 ,F,0241,KFLL,08:28:43,13:16:16, 70 ,F,0242,TJSJ
> AC#1,N39216,64%,12HR,IFR,04:41:26,07:31:09, 80 ,F,6432,KPWK,10:41:15,13:30:58, 90 ,F,6431,KHEF
> AC#1,N39216,64%,12HR,IFR,04:41:26,07:31:09, 80 ,F,6432,KPWK,10:41:20,13:31:03, 90 ,F,6431,KHEF
> AC#1,N39336,26%,12HR,IFR,02:26:31,07:04:40, 110 ,F,3163,OMAD,08:26:26,13:04:35, 120 ,F,3164,OYAA
>
> AC#1,N39368,82%,6HR,VFR,00:25:40,02:12:30, 115 ,F,7827,RPVM,03:25:40,05:12:30, 125 ,F,7828,RPLL
> AC#1,N39368,82%,6HR,VFR,00:25:40,02:12:30, 115 ,F,7827,RPVM,03:25:42,05:12:32, 125 ,F,7828,RPLL note how these three should be dupes
> AC#1,N39368,82%,6HR,VFR,00:25:42,02:12:32, 115 ,F,7827,RPVM,03:25:45,05:12:35, 125 ,F,7828,RPLL
>
> AC#1,N39416,65%,12HR,IFR,05:04:22,08:37:13, 60 ,F,2365,KTVF,11:04:38,14:37:29, 50 ,F,2366,KMIE
> AC#1,N39416,65%,12HR,IFR,05:04:27,08:37:18, 60 ,F,2365,KTVF,11:04:38,14:37:29, 50 ,F,2366,KMIE
> AC#1,N39416,65%,12HR,IFR,05:04:32,08:37:23, 60 ,F,2365,KTVF,11:04:38,14:37:29, 50 ,F,2366,KMIE
> AC#1,N39448,66%,12HR,VFR,04:31:40,07:39:16, 125 ,F,2858,VAJJ,10:31:35,13:39:11, 115 ,F,2859,VOCI
> AC#1,N39456,80%,8HR,IFR,00:16:28,02:46:59, 90 ,F,7893,YGLI,04:16:35,06:47:06, 80 ,F,7894,YMAY
> AC#1,N39456,80%,8HR,IFR,00:16:31,02:47:02, 90 ,F,7893,YGLI,04:16:35,06:47:06, 80 ,F,7894,YMAY
> AC#1,N39552,34%,8HR,IFR,02:15:53,03:58:39, 110 ,F,6086,KRYN,06:15:46,07:58:32, 120 ,F,6085,61B
> AC#1,N39552,34%,8HR,IFR,02:15:53,03:58:39, 110 ,F,6086,KRYN,06:15:49,07:58:35, 120 ,F,6085,61B
> AC#1,N39568,24%,24HR,IFR,00:52:55,06:35:14, 100 ,F,7790,KGCC,12:52:55,18:35:14, 110 ,F,7789,KHTS
> AC#1,N39576,75%,8HR,IFR,00:58:50,01:56:28, 90 ,F,2870,LGMT,04:58:50,05:56:28, 100 ,F,2869,LGAV
> AC#1,N39576,75%,8HR,IFR,00:58:53,01:56:31, 90 ,F,2870,LGMT,04:58:50,05:56:28, 100 ,F,2869,LGAV
> AC#1,N3960,81%,8HR,VFR,00:47:21,03:03:49, 75 ,F,6969,LKKU,04:47:14,07:03:42, 85 ,F,6968,EDDW
> AC#1,N3960,81%,8HR,VFR,00:47:21,03:03:49, 75 ,F,6969,LKKU,04:47:17,07:03:45, 85 ,F,6968,EDDW
> AC#1,N39656,20%,12HR,IFR,04:57:04,10:02:18, 90 ,F,3957,LIRN,10:57:15,16:02:29, 100 ,F,3956,HLGT
> AC#1,N39656,80%,24HR,IFR,00:10:33,02:18:18, 110 ,F,9911,SBES,12:10:12,14:17:57, 100 ,F,9910,SBUR
> AC#1,N39656,80%,24HR,IFR,00:10:33,02:18:18, 110 ,F,9911,SBES,12:10:22,14:18:07, 100 ,F,9910,SBUR
> AC#1,N3984,24%,12HR,IFR,01:49:10,06:24:49, 80 ,F,9723,UHMA,07:48:59,12:24:38, 90 ,F,9724,PAFA
> AC#1,N39912,78%,12HR,IFR,04:55:19,07:42:17, 80 ,F,5352,KFSM,10:55:19,13:42:17, 90 ,F,5351,KPWK
> AC#1,N39912,78%,12HR,IFR,04:55:24,07:42:22, 80 ,F,5352,KFSM,10:55:19,13:42:17, 90 ,F,5351,KPWK
> AC#1,N39944,62%,8HR,VFR,02:02:21,05:15:12, 55 ,F,1467,KPYM,06:02:21,09:15:12, 65 ,F,1466,KCPC
> AC#1,N39944,62%,8HR,VFR,02:02:24,05:15:15, 55 ,F,1467,KPYM,06:02:21,09:15:12, 65 ,F,1466,KCPC
> AC#1,N39952,30%,6HR,IFR,00:22:01,02:33:26, 120 ,F,0785,MHTG,03:22:04,05:33:29, 110 ,F,0784,MMCZ
> AC#1,N40008,52%,12HR,IFR,01:38:37,04:43:46, 110 ,F,8274,LGKA,07:38:37,10:43:46, 120 ,F,8273,LIEE
> AC#1,N40008,52%,12HR,IFR,01:38:42,04:43:51, 110 ,F,8274,LGKA,07:38:37,10:43:46, 120 ,F,8273,LIEE
> AC#1,N40128,7%,24HR,IFR,05:41:11,10:25:31, 60 ,F,1008,KSTP,17:41:01,22:25:21, 70 ,F,1009,KFRG
> AC#1,N40160,48%,6HR,VFR,02:28:27,04:26:57, 75 ,F,7104,KALN,05:28:22,07:26:52, 65 ,F,7105,KBEC
> AC#1,N40224,48%,8HR,IFR,00:29:53,03:43:00, 60 ,F,3925,KHUM,04:29:53,07:43:00, 50 ,F,3926,MUHA
> AC#1,N40408,6%,24HR,IFR,08:41:01,13:47:23, 80 ,F,7929,YBTL,20:41:01,01:47:23, 90 ,F,7930,YSCB
> AC#1,N40448,26%,6HR,IFR,00:54:35,03:08:37, 90 ,F,8121,ESOK,03:54:35,06:08:37, 80 ,F,8122,EDWI
> AC#1,N40464,83%,12HR,VFR,02:40:35,06:12:00, 75 ,F,5207,KMCB,08:40:40,12:12:05, 85 ,F,5208,KHOB
> AC#1,N40528,71%,6HR,IFR,00:56:02,01:34:29, 50 ,F,4694,3T1,03:56:02,04:34:29, 40 ,F,4695,KDFW
> AC#1,N40528,71%,6HR,IFR,00:56:02,01:34:29, 50 ,F,4694,3T1,03:56:04,04:34:31, 40 ,F,4695,KDFW
> AC#1,N40528,71%,6HR,IFR,00:56:04,01:34:31, 50 ,F,4694,3T1,03:56:07,04:34:34, 40 ,F,4695,KDFW
> AC#1,N40576,93%,8HR,IFR,02:43:15,05:57:46, 60 ,F,8550,Q24,06:43:15,09:57:46, 70 ,F,8551,KBTR
> AC#1,N40576,93%,8HR,IFR,02:43:15,05:57:46, 60 ,F,8550,Q24,06:43:18,09:57:49, 70 ,F,8551,KBTR
> AC#1,N40680,18%,6HR,IFR,02:15:24,04:17:20, 110 ,F,2025,MTPP,05:15:18,07:17:14, 120 ,F,2026,MUCM
> AC#1,N40712,12%,8HR,IFR,00:28:08,01:52:21, 120 ,F,0841,KEUG,04:28:08,05:52:21, 110 ,F,0842,KPAE
> AC#1,N40800,25%,12HR,VFR,05:47:10,10:35:38, 115 ,F,9694,OMAL,11:47:05,16:35:33, 125 ,F,9695,OYAA
> AC#1,N40800,61%,8HR,IFR,02:23:02,04:04:03, 120 ,F,8933,KSCK,06:22:55,08:03:56, 110 ,F,8934,KLGB
> AC#1,N40816,58%,12HR,IFR,00:56:52,05:34:40, 80 ,F,0160,KTYR,06:56:57,11:34:45, 90 ,F,0159,KBCT
>
> Windows Live Hotmail is giving away Zunes. Enter for your chance to win.
