Here, I shall be updating relevant progress made in NLP-Engine profiling.
@dehilster @meyersamnon
1 Like
For the first week and a half, I managed to compile and build the NLP-Engine from the repo. I faced some issues along the way related to icu and vcpkg. Fixed by installing autoconf-archive package.
As far as profilers are concerned, I tried to use valgrind, because I’ve worked with it previously. Using it I’m able to get the cumulative CPU instructions executed by each module, a fair idea of resources used. I’m thinking total time by each module is a better indicator, so my plan is to explore other profilers like gperftools and the one from intel to see if they are any better.
1 Like
Thanks, Niranjan, sounds like a good start.
Would be good to know what analyzer(s) you are testing on, as we may want a suite that focuses on different parts of the NLP++ runtime. Eg, analyzers that focus on KB, on dictionary, on pattern passes, on recursive grammar passes, and so on. Some small analyzers can test these.
And some large analyzers and large KBs/dictionaries can show where scaling up causes bottlenecks.
On that note, I just had a quick question about analyzers.
I was curious on what KBs, KBBs are in general. When i go through the file, I see that they are some kind of reference materials? And how do I go about using the analyzers repository under @dehilster you have for use with the engine.
I can figure out the second part eventually, but wanted to know what the repository has and how they fit in place with the entire NLP++.
They are knowledge base files. They are a hierarchical database that NLP++ uses for knowledge. It is known as the conceptual grammar. The KB files are old, non-human readable files that were used to dump and read in knowledge. The KBB files are the new human-readable files that are now used.
1 Like
Actually .dict files and .kbb files end up in the same place: the conceptual grammar (or knowledge base). These files are used to make it easy for users to create dictionaries and knowledge.
1 Like