libdwfl: Don't allocate more than SIZE_MAX in dwfl_segment_report_module.

The code in dwfl_segment_report_module tries to allocate and fill in
memory as described in a core file. Normally all memory in filled in
through the (phdrs) memory_callback or the read_eagerly callback. If
the last callback doesn't work we try to calloc file_trimmed_end bytes
and then try to fill in the parts of memory we can from the core file
at the correct offsets.

file_trimmed_end is a GElf_Off which is an unsigned 64bit type. On
32bit systems this means when cast to a size_t to do an allocation
might allocate truncated (much smaller) value. So make sure to not
allocate more than SIZE_MAX bytes.

It would be nice to have a better way to limit the amount of memory
allocated here. A core file might describe really big memory areas for
which it doesn't provide any data. In that case we really shouldn't
calloc mega- or giga-bytes of zeroed out memory.

Reported-by: Evgeny Vereshchagin <evvers@ya.ru>
Signed-off-by: Mark Wielaard <mark@klomp.org>

2 files changed, 8 insertions, 0 deletions

diff --git a/libdwfl/ChangeLog b/libdwfl/ChangeLog
index 856bd335..aaea164c 100644
--- a/libdwfl/ChangeLog
+++ b/libdwfl/ChangeLog
@@ -1,3 +1,8 @@
+2021-12-12  Mark Wielaard  <mark@klomp.org>
+
+        * dwfl_segment_report_module.c (dwfl_segment_report_module): Don't
+        allocate more than SIZE_MAX.
+
 2021-12-09  Mark Wielaard  <mark@klomp.org>
 
         * link_map.c (dwfl_link_map_report): Limit dyn_filesz malloc size
diff --git a/libdwfl/dwfl_segment_report_module.c b/libdwfl/dwfl_segment_report_module.c
index f4acfe53..46564ec5 100644
--- a/libdwfl/dwfl_segment_report_module.c
+++ b/libdwfl/dwfl_segment_report_module.c
@@ -904,6 +904,9 @@ dwfl_segment_report_module (Dwfl *dwfl, int ndx, const char *name,
       /* The caller wants to read the whole file in right now, but hasn't
          done it for us.  Fill in a local image of the virtual file.  */
 
+      if (file_trimmed_end > SIZE_MAX)
+        goto out;
+
       void *contents = calloc (1, file_trimmed_end);
       if (unlikely (contents == NULL))
         goto out;